Developing Windows 7 based Multi-Touch Application
Ramaprasanna ChellamuthuMicrosoft India
Agenda
Touch HardwareKey Touch ScenariosWindows 7 Touch InvestmentsTouch Application Development Guidance
GestureManipulation and Inertia
DemoResources
Multi-Touch…
…creates new opportunities.Software that is intuitive for users to walk up & masterSoftware that is collaborative with multiple users
…is not new.Researchers have been exploring this stuff for decades
…is now going mainstream.1. Hardware: Robust touch-capable hardware from Microsoft and OEMs
1. Dozens of Win7 touch devices now available2. OS: Windows 7 has great touch support baked in3. SDKs: Microsoft is making touch easy to leverage4. You: Developers & designers creating innovative apps that take
advantage of the technology
Windows Touch is hereWindows Touch is defined as two or more independently traceable touch points.
ConsumersTouch interfaces have proven successful Multi-Touch offers clear differentiation with a high “WOW” factor Multi-touch enhances on-the-go use and enable new scenarios
HardwareMulti-touch capable machines in market today in a broad set of form factors
Navigating and Consuming the Web
Playing Casual Games
Reading and Sorting Email
Viewing Photos
1
2
3
4
5
6 Navigating Files and Using Applications
Consuming Music and Video
Key Touch Scenarios Are Consumptive
Windows InvestmentInternet Explorer®
Pan and zoom gestures, flick forward/back“Open link in new tab” gestureAddress bar drag menu
Windows® Media Center:Live gallery panningOn-screen keyboardLarger transport controls & seekNew Now Playing & details pages
Photo Viewer (& Windows Live Photo Gallery)Multitouch Zoom and rotation
XPS ViewerStretch zoom centered around gestureSmart zoom
Touch Pack for Windows 7
Windows 7 Principle: Be Great at “Look” and “Do”
Users have expectations for how an experience will “Look” and what it should “Do”Direct Manipulation demands more from “Look” and “Do”Touch Pack for Windows 7
Surface Globe: Immersive, 2-D and 3-D touch world exploration:
Zoom in on locationsPinpoint areas of interestGet local informationPersonalize map views
Development TiersBest
Better
Good
Raw data points from WM_TOUCH for custom gestures. Manipulation/inertia APIs
Multi-touch support – zoom, finger pan, rotate, panning with inertia, and 2 finger tap. WM_GESTURE
Maps to common Windows messages – pan, zoom, right click, and double click
Gestures: Consistency is key
Gestures: Touch actions that are one step removed from direct manipulations
Two finger tapPress and tap Pinch/stretch (zoom)Circular motion (rotate)Single and two-finger panningFlicks
Useful for shortcut-like efficiencyConsistent operations like zoom in, or navigateFocus on content : to surface extra information or invoke contextual action UI
Design ConsiderationsConsistency
Should work in the same way across Windows applicationsKeep it simple – users won’t discover or remember too many abstractions
Windows 7 Touch Gestures
WM_GESTURE
Windows Message and APIs:
WM_GESTURE, WM_GESTURENOTIFY
GESTURECONFIG struct
Set/GetGestureConfig()
GESTUREINFO
GetGestureinfo()
Setting Gestures and WM_GESTURE case WM_GESTURENOTIFY: { BOOL bResult;
GESTURECONFIG gcAll[] = {0,GC_ALLGESTURES,0}; // Enable all gestures bResult = SetGestureConfig(hWnd, 0, 1, gcAll, sizeof(GESTURECONFIG));
} case WM_GESTURE: {
GESTUREINFO gi; gi.cbSize = sizeof(gi); bResult = GetGestureInfo((HGESTUREINFO)lParam, &gi); switch (gi.dwID) { case GID_BEGIN:
break; case GID_END:
break; case GID_ZOOM: //GID_PAN, GID_ROTATE, GID_TWOFINGERTAP, GID_PRESSANDTAP
break; }
WM_TOUCH
WM_TOUCH* family of messagesEnables touch-optimized experiencesProvides standard Win32 messages
WM_TOUCHDOWN, WM_TOUCHMOVE, WM_TOUCHUPSemantically similar to mouse messagesConveys raw touch data to Win32 apps
Scenario examples:Finger painting, custom gestures, feeding higher level controls, etc
.NET 3.5 Multitouch library
Multi-touch for Windows 7
NativeWindows SDK
.NET 4WPF
SilverlightSilverlight3
Gestures Available via manipulation
Raw Touch
Manipulation
Inertia
Manipulation Container
Canvas
Manipulation Container
IsManipulationEnabled=true
IsManipulationEnabled=true
Update RenderTranform
Handle ManipulationDelta Event
• Coordinates relative to Manipulation Container
Handle ManipulationStarting Event• Set Manipulation
Container
Manipulation Events
Starting Started Delta Inertia Starting Delta Complete
d
StartInertia() StartInertia()
Touch Down (Initial) Touch Move Touch Up
(All)
Initialize:•Mode• Container• Pivot
Completed()
WPF 4 Multi-Touch APIs Overview
WPF Controls Styles with Panning Enabled
ScrollViewer Panning Support
Manipulation Events
Touch Events
Extensible Touch Device
Win7 Touch Device Surface Touch Device
Beta 1
Beta 2
Release Candidate
Surface V2
Silverlight 3 Touch APIs - Listening
public partial class MainPage : UserControl { public MainPage() { ... // listen to touch events from the system Touch.FrameReported += new TouchFrameEventHandler(OnFrame); }
void OnFrame(object sender, TouchFrameEventArgs e) { // enumerate and respond to touch events }}
Silverlight 3 Touch APIs - Processing
void Touch_FrameReported(object sender, TouchFrameEventArgs e) { TouchPointCollection touchPoints = e.GetTouchPoints(LayoutRoot);
foreach (TouchPoint tp in touchPoints) { if (tp.Action == TouchAction.Down) { // a new touch has come down } if (tp.Action == TouchAction.Move) { // a previously down touch has moved } if (tp.Action == TouchAction.Up) { // a touch has been removed } }}
Inertia BehaviorsManipulationInertiaStartingEventArgs
Properties Units
TranslationBehavior InitialVelocity 1/96th DIP per millisec
DesiredDeceleration 1/96th DIP per millisec^2
DesiredDisplacement 1/96th DIP
RotationBehavior InitialVelocity Degrees per millisec
DesiredDeceleration Decrees per millisec^2
DesiredRotation Degrees
ExpansionBehavior InitialVelocity 1/96th DIP per millisec
InitialRadius 1/96th DIP
DesiredDeceleration 1/96th DIP per millisec^2
DesiredExpansion 1/96th DIPS
• Deceleration and Displacement/Rotation/Expansion mutually exclusive• Deceleration – useful to simulate friction• Displacement – useful on page flips• Default – no inertial movements
Demo !! Demo !! Demo !!
Surface is not just multi-touch
Windows 7 & WPF 4 are a great baseline for touch
Surface creates unique opportunities with specialized HW
Robust topHorizontal form factorInfrared camera array
Surface: Beyond Multi-Touch
Massive multi-touch
Multi-user experiences
Object recognition
Optical engineering
Surface: Multi-User
Trend: Software continues to reduce the need for face-to-face human interactions
Problem: Happiness and creativity depend on social interactions
Surface brings people togetherIndustrial design encourages users to gatherInput capabilities allow simultaneous useApps facilitate collaboration / competition
Surface: Massive Multi-Touch
Surface can efficiently track 50+ simultaneous fingers and objects
Surface: Objects
Uniquely identify & react to billions of tagged objects
Byte Tags256 unique values
Identity Tags>340,282,366,920,938,000,000,000,000,000,000,000,000
unique values
Tags include orientation indictors dots
Surface: Optics
Determine the size and shape of every inputSee into and project onto physical objects
>>FUTURE
What’s next for Surface hardware?
Things we’re exploring:Cheaper ThinnerVertical
Stay tuned…
ResourcesSoftware Design
Windows User Experience Interaction GuidelinesMulti-Touch in Windows 7Developer Enhancements to Windows Touch and Tablet PC(User Perceptions of Response Time) Performance UX Patterns
Developer ResourcesCode Magazine: Multi-touch capabilities in Windows 7Windows Touch on MSDNWindows Touch Developer ResourcesTablet and Touch SDKIntroducing the Microsoft Touch Pack for Windows 7
Resources
www.microsoft.com/teched
Sessions On-Demand & Community Microsoft Certification & Training Resources
Resources for IT Professionals Resources for Developers
www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
Learning
Thank you !!