EVALUATING CUES FOR RESUMING INTERRUPTED PROGRAMMING

TASKSChris Parnin, Georgia Institute of Technology

Robert DeLine, Microsoft Research

Interruptions force rapid suspension of task memory --

which, for programmers, is large and complex.

But for programmers, task memory is so large, even a

lunch break can be a problem.

INTERRUPTIONS

A programmer makes several changes, one of which contains an error.

Goes to lunch

Running code crashes, but forgot “what was changed”?

Spends 15 minutes trying to remember.

EXAMPLE

Managers 100s tasks with few documents.

Programmers Few tasks with 100s documents.

TASKS

Typical program with 56 sub-projects.Note: sub-folders not expanded.

DOCUMENTS

What do programmers forget and how can we remind them?

RESEARCH GOAL

1) Survey programmers to identify current strategies and new approaches.

2) Experimentally evaluate current strategies, and compare with current research approaches.

RESEARCH APPROACH

15 questions composed of both fixed and open-responses.

371 developers from Microsoft and 42 from various companies responded.

SURVEY

MarkersReminders

TOP PROSPECTIVE MEASURE:NOTE-TAKING

S.H.

RESUMPTION STRATEGIES

Return to last method modified, and navigate to related code to jog your memory.

Use markers or notes.

View source code difference.

Run program and examine its output or UI.

RESUMPTION STRATEGIES

Return to last method modified, and navigate to related code to jog your memory.

Use markers or notes.

View source code difference.

Run program and examine its output or UI.

cue-seeking, searching for reminders while reactivating mental

context.

10-15 minutes before first edit(based on recorded sessions)

HELP!

Programmers wanted:

Better help for viewing work history: (recent code changes, documents, summaries)

Didn’t want as much:

Instant replays, task planning tools, smart reminders.

WORK HISTORY

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

Record document switches, actions.

Record state of document after each action.

Uses semantic cues.

www.eclipse.org/mylyn

CURRENT APPROACH

SpaceInvadersSpaceInvaders

ReferencesApp.icoAssemblyInfo.csBomb.csBullet.csForm1.cs

Form1.resxForm1

TestBulletCollisionMoveInvadersHandleKeystimer1_Tick

GameObject.csHighScore.csInvader.csInvaderRow.cs

InvaderRowCollisionTestMoveDown

LivesIndicator.csMan.cs

ManGetBulletStart

Saucer.cs

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

Use both episodic and semantic cues.

ALTERNATIVE APPROACH var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

var first = line.Points.First(); var last = line.Points.Last();

return new Point((first.X + last.X) / 2, (first.Y + last.Y) / 2);

xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

var last = line.Points.Last();

SAVE

EXPERIMENT

Do semantic and episodic-based cues from work history help resumption?

PARTICIPANTS

15 participants (one female only)

Age (range 31 to 56) [39 avg.]

Professional developers working in Washington Puget Sound area.

Make block fall all the way down with space key.

Make ghosts stop moving when eating power pellet.

Make monster reappear if missed shot.

TASKS

METHODParticipant in 2 hour session,

beginning with training on tools.

Programmers interrupted shortly after making edit,moving on to next task.

Programmers use assigned condition to help resume

task(could take notes).

RESULTS

• Near 2-to-1 task completion rate (either cues) over notes. (chi, p < 0.1)

– 7,7,4

• Near 2-to-1 resumption speed (DOI) over notes. (no significance)

– 2.5, 3.5, 4.5 minutes

• Subjective ratings highest for timeline, notes, then DOI. (t-test, p <0.001)– Next slide

RATINGS

0.0

1.0

2.0

3.0

4.0

5.0

speed details durability overall

DOI

content

note

Participants completing no tasks, rated notes highest:

Experienced programmers create better history.

“There is something wrong, I don’t know what I missed”

this.keyUp = keyUpHandler;

FORGETTING

5 minutes pass…but too little time to finish.

“I remember finding that code, I just can’t remember where it is!”

FORGETTING

NOTES

Notes restore goals well, but context poorly.

DOI TREEVIEW

• Fast, but incomplete.

• Seemed to support familiarity memory, clicking on locations without remembering what’s there.

• “I wouldn’t remember this if I had to come back in 5 minutes”

SpaceInvadersSpaceInvaders

ReferencesApp.icoAssemblyInfo.csBomb.csBullet.csForm1.cs

Form1.resxForm1

TestBulletCollisionMoveInvadersHandleKeystimer1_Tick

GameObject.csHighScore.csInvader.csInvaderRow.cs

InvaderRowCollisionTestMoveDown

LivesIndicator.csMan.cs

ManGetBulletStart

Saucer.cs

CONTENT TIMELINE

• Slower, but restorative.

• Users worked backwards in their task state with this view.

• Seemed to support restoring episodic and prospective memory.

• “The history of the [timeline] and diffs are very useful and did the most help in restarting”

COMMENTS AND FEEDBACK

“I wouldn’t have to write down the location, just what needs to be done”

DOI Treeview should explain how I used item:Edit, Find, Navigation, Debug Step-through

Content Timeline should include:filtering, pinning, annotation, and sharing.

BEYONDExcel/Word Summaries

Sharing Work Experiences

FUTURE

Episodic Buffer Monitor

Pen and multitouch-based interactions and annotations

Temporal and semantic zoom

Subvocalization (inner voice)

QUESTIONS

EVALUATING CUES FOR RESUMING INTERRUPTED PROGRAMMING

TASKSChris Parnin, Robert DeLine

Task Completion

Task 1 Task 2 Task 3 Total

DOI treeview

3 2 2 7

Content timeline

1 4 2 7

Notes-only

1 2 1 4

Total 5 8 5

Failure Causes

Problem Solving

Relocation

Mistakes Total

DOI treeview

1 0 1 2

Content timeline

2 0 0 2

Notes-only

1 2 2 5

Total 4 2 3