Ken HinckleyPrincipal ResearcherMicrosoft Research

Michel PahudBill Buxton

Direct Display Interaction via Simultaneous Pen and

Multi-Touch Input

• Handhelds• Slates• Drafting Boards• Tabletops• Wall Displays

• Touch, or• Pen.

DIRECT INPUT A SHIFT TO

ISN’T THIS ABOUTTHE FUTURE OF TOUCH?

WHY THE PEN?

To help explain, let me introduce my nemesis.

MR. I.M.A. BIGBODY, CORPORATE DENIZEN,THIRD RATE INC.

“I CAN TYPE FASTERTHAN I CAN WRITE.”

TYPETHIS.

OR THIS.

OR MAYBETHIS.

YOU CAN’T TYPE THE SOUL OFA NEW IDEA.

BUT IT’S NOT ALL ABOUTTHE PEN.

FIRST YOU NEED TO GRAB THE NAPKIN.

“JUST TELL ME WHICH IS BEST, TOUCH OR PEN.”

PROPERTIES SHARED BY

PEN AND TOUCH

PEN IS BEST!

Mechanical intermediary: familiar tool, lever arm, affords precision!

TOUCH IS BEST!

No mechanical intermediary!

Nothing to lose!

No time to unsheathe!

PEN IS BEST!

Definite, intentional pressure to leave a mark

TOUCH IS BEST!

Just rest your hand…

No force required

BUT…“Midas Touch”

EVERY COIN HAS TWO SIDES.

OF HANDSAND

HANDEDNESS

Q: Which hand do you write with, Right or Left?

A: Yes

• Guiard, 1987: NPH defines frame of reference.• Right vs. Left is the wrong question to ask.

What is the logic of the division of labor

between pen and touch in interface design?

Pen WritesTouch Manipulates

fluidity of annotation + manipulation navigation zooming object creation

Pen + Touch Yields New Tools

• Video• Dell XT2 Demo

PEN + TOUCH GESTURES

SUMMARY

IF YOU’RE ONLY THINKING ABOUT TOUCH, YOU’RE BLOWING IT.

GET OFF THE MULTI-TOUCHESCALATOR.

Not pen. Not touch.

Not pen OR touch.Pen AND Touch,

designed in concert

“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,

SO WHAT’S THE BIG DEAL?”

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,

SO WHAT’S THE BIG DEAL?”