Date post: | 23-Dec-2015 |
Category: |
Documents |
Upload: | bridget-johnston |
View: | 213 times |
Download: | 1 times |
Tablet PC Platform Advanced Topics
Tablet PC Platform Advanced Topics
Software Design Engineer/Test LeadSoftware Design Engineer/Test [email protected]@microsoft.com
TopicsTopics
1.1. Tablet Hardware API Tablet Hardware API
2.2. Overview of the Tablet SDKOverview of the Tablet SDKa.a. Ink, Stroke and related objectsInk, Stroke and related objects
b.b. RendererRenderer
3.3. Clipboard SupportClipboard Support
4.4. Ink Recognition Ink Recognition
5.5. Gesture SupportGesture Support
Tablet PC Platform Hardware APITablet PC Platform Hardware API
Examining Tablet HardwareExamining Tablet Hardware
Electromagnetic digitizersElectromagnetic digitizersHigh resolution, High throughputHigh resolution, High throughput
There are two types of digitizersThere are two types of digitizersIntegrated and Non-IntegratedIntegrated and Non-Integrated
Multiple Digitizers are supportedMultiple Digitizers are supportedThe mouse is also represented as a digitizerThe mouse is also represented as a digitizer
Digitizers support various capabilitiesDigitizers support various capabilities# of stylus buttons, pressure, angle or rotation # of stylus buttons, pressure, angle or rotation attributesattributes
Ink Stroke PacketData varies between digitizersInk Stroke PacketData varies between digitizers
Tablet Hardware APITablet Hardware API
TabletTabletObject representation of a digitizerObject representation of a digitizer
Query for digitizer’s supported properties Query for digitizer’s supported properties
Get property metricsGet property metricsE.g.: How many levels of pressure are E.g.: How many levels of pressure are supported?supported?
TabletsTabletsReturns collection of all connected Returns collection of all connected digitizersdigitizers
Including the mouseIncluding the mouse
Tablet Hardware API Cont.Tablet Hardware API Cont.
CursorCursorRepresents a stylusRepresents a stylus
Allows for access to stylus propertiesAllows for access to stylus propertiesInverted, Button state, NameInverted, Button state, Name
Available in all InkCollector/Overlay Available in all InkCollector/Overlay eventsevents
CursorInRange / CursorOut0fRangeCursorInRange / CursorOut0fRange
CursorDown / CursorUpCursorDown / CursorUp
CursorsCursorsCollection of Cursor objectsCollection of Cursor objects
Ink Overlay & Ink CollectorInk Overlay & Ink Collector
Main function is to collect inkMain function is to collect ink
Ink Collector provides core Ink Collector provides core functionalityfunctionality
Ink Overlay is an Ink Collector with the Ink Overlay is an Ink Collector with the addition of easy to use Editing Modes addition of easy to use Editing Modes (Ink, Select, Erase)(Ink, Select, Erase)
Contains an Ink objectContains an Ink object
Full set of event notificationsFull set of event notifications
InkCollectorInkCollectorTop level object for collecting ink Top level object for collecting ink
Attaches to any HWNDAttaches to any HWND
Handles input from all attached Tablets Handles input from all attached Tablets (digitizers) and Cursors (pens)(digitizers) and Cursors (pens)
Intercepts cursor movementsIntercepts cursor movements
CollectionMode propertyCollectionMode propertyInk OnlyInk Only
Gesture OnlyGesture Only
Ink and GestureInk and Gesture
InkOverlayInkOverlay
Superset of InkCollectorSuperset of InkCollector
Adds Editing ModesAdds Editing ModesInk – Real-time inking modeInk – Real-time inking mode
Select – Selection modeSelect – Selection mode
Delete – Eraser mode (both point and stroke)Delete – Eraser mode (both point and stroke)
Ink Collection EventsInk Collection EventsCursorInRange/CursorOutOfRangeCursorInRange/CursorOutOfRange
NewInAirPacketsNewInAirPackets
CursorDownCursorDown
NewPacketsNewPackets
StrokeStroke
GestureGesture
SystemGestureSystemGesture
Mouse EventsMouse Events
Ink ObjectInk Object
Primary Data Object in the Tablet Primary Data Object in the Tablet PlatformPlatform
Container for:Container for:Stroke dataStroke data
MetadataMetadata
Controls all means of persistenceControls all means of persistence
Extensible via ExtendedProperties Extensible via ExtendedProperties collectioncollection
Stroke ObjectStroke ObjectRepresents a single strokeRepresents a single stroke
Single pen down, pen move, pen up Single pen down, pen move, pen up sequencesequence
ExtendedPropertiesExtendedPropertiesAllows applications to access and store Allows applications to access and store custom datacustom data
DrawingAttributesDrawingAttributesControls the rendering of the strokeControls the rendering of the stroke
Rich API Rich API PacketData, BezierPoints, PacketData, BezierPoints,
PolylineCusps, SelfIntersectionsPolylineCusps, SelfIntersections
Strokes CollectionsStrokes CollectionsWhat are Stroke collections?What are Stroke collections?
Collections of references to strokesCollections of references to strokes
Not the stroke data itselfNot the stroke data itself
Important so you can pass a subset to the Important so you can pass a subset to the RecognizersRecognizers
Stroke Collections can share strokesStroke Collections can share strokesBoth Stroke collections include Stroke 3Both Stroke collections include Stroke 3
Strokes
Stroke (1)
Strokes
Stroke (2)Stroke (3)Stroke (4)Stroke (5)
DrawingAttributes ObjectDrawingAttributes Object
Controls the appearance of InkControls the appearance of InkColor, width, transparencyColor, width, transparency
Smoothing, anti-aliasingSmoothing, anti-aliasing
Height, widthHeight, width
Pen Tip: Ball or Rectangle Pen Tip: Ball or Rectangle
DefaultDrawingAttributes property DefaultDrawingAttributes property Specifies the DrawingAttributes for all CursorsSpecifies the DrawingAttributes for all Cursors
Can be applied to Can be applied to StrokeStroke
Strokes collectionStrokes collection
Cursor (Pen)Cursor (Pen)
Renderer ObjectRenderer Object
Two coordinate systems on a Tablet PCTwo coordinate systems on a Tablet PCDevice coordinatesDevice coordinates
Ink coordinates (HIMETRIC units)Ink coordinates (HIMETRIC units)
Renderer objectRenderer objectMap between coordinate systemsMap between coordinate systems
Draw methods for renderingDraw methods for rendering
Manipulation methodsManipulation methodsObject and View transforms supportedObject and View transforms supported
Helper methods: Scale, Move, RotateHelper methods: Scale, Move, Rotate
Clipboard SupportClipboard Support
Clipboard SupportClipboard SupportUse the Ink API to copy and pasteUse the Ink API to copy and paste
ClipboardCopyClipboardCopy
ClipboardPaste ClipboardPaste
Multiple formats supportedMultiple formats supportedISF – Ink Serialized FormatISF – Ink Serialized Format
HTMLHTML
Bitmap, Metafile Bitmap, Metafile
Applications interacting with the Applications interacting with the clipboard can choose the formatclipboard can choose the format
Ink RecognitionInk RecognitionInterpret pen movements and/or strokes asInterpret pen movements and/or strokes as
Text (Handwriting)Text (Handwriting)
GesturesGestures
Shapes and symbolsShapes and symbols
Language SupportLanguage Support
Synchronous and Asynchronous Synchronous and Asynchronous recognition supportedrecognition supported
Provides a full set of recognition resultsProvides a full set of recognition results
Extensibility architectureExtensibility architecture
Recognizer ContextRecognizer ContextObject for accessing handwriting Object for accessing handwriting recognizersrecognizers
Binds ink data to a particular language for Binds ink data to a particular language for recognitionrecognition
Performs synchronous or asynchronous Performs synchronous or asynchronous handwriting recognitionhandwriting recognition
Recognition results and alternates Recognition results and alternates returnedreturned
Attributes used to bias recognition resultsAttributes used to bias recognition results
RecognitionResultRecognitionResultRecognizers contain a set of Recognizers contain a set of structuresstructures
Each structure called a “lattice”Each structure called a “lattice”
TopString propertyTopString property
TopConfidence propertyTopConfidence property
TopAlternate propertyTopAlternate property
RecognitionAlternates objectRecognitionAlternates objectGetAlternatesFromSelection methodGetAlternatesFromSelection method
Access to alternate paths through the Access to alternate paths through the latticelattice
Gesture SupportGesture Support
GesturesGestures
Collected through InkCollector, Collected through InkCollector, InkOverlay, or RealTimeStylusInkOverlay, or RealTimeStylus
Two Types: System or ApplicationTwo Types: System or Application
System GesturesSystem GesturesMimic mouse eventsMimic mouse events
Cannot be disabledCannot be disabled
Application GesturesApplication GesturesCan be single or multi-strokeCan be single or multi-stroke
Can be turned on/off individuallyCan be turned on/off individually
System GesturesSystem Gestures
Available GesturesAvailable GesturesTap Tap
DoubleTapDoubleTap
RightTapRightTap
DragDrag
RightDragRightDrag
HoldEnterHoldEnter
HoverEnterHoverEnter
HoverLeaveHoverLeave
Coordinates are Window CoordinatesCoordinates are Window Coordinates
Can’t be cancelled Can’t be cancelled
Application GesturesApplication Gestures
CollectionMode determines supportCollectionMode determines supportInkOnlyInkOnly
InkAndGestureInkAndGesture
GestureOnlyGestureOnly
Single vs. Multi-Stroke GesturesSingle vs. Multi-Stroke Gestures
Inform your users about gesture Inform your users about gesture supportsupport
Good UI feedbackGood UI feedback
Consistent use (i.e. Scratchout)Consistent use (i.e. Scratchout)
GesturesGesturesFocus:Focus:•Scenario driven: don’t over Scenario driven: don’t over useuse
The Tablet and Mobile PC The Tablet and Mobile PC Developer CenterDeveloper Center
http://msdn.microsoft.com/mobilepc
Tablet PC Developer CenterTablet PC Developer Center
The latest technical articlesThe latest technical articles
DownloadsDownloads
Developer resourcesDeveloper resources
Regular columnsRegular columns
..
NewsgroupsNewsgroups
Featured partnersFeatured partners
and more……and more……
Attend a free chat or web castAttend a free chat or web casthttp://www.microsoft.com/communities/chats/default.mspxhttp://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asphttp://www.microsoft.com/usa/webcasts/default.asp
List of newsgroupsList of newsgroupshttp://communities2.microsoft.com/communities/newsgroups/en-us/default.aspxhttp://communities2.microsoft.com/communities/newsgroups/en-us/default.aspx
MS Community SitesMS Community Siteshttp://www.microsoft.com/communities/default.mspxhttp://www.microsoft.com/communities/default.mspx
Locate Local User Groups and Community SitesLocate Local User Groups and Community Siteshttp://www.microsoft.com/communities/usergroups/default.mspxhttp://www.microsoft.com/communities/usergroups/default.mspx http://www.microsoft.com/communities/related/default.mspxhttp://www.microsoft.com/communities/related/default.mspx
© 2005 Microsoft Corporation. All rights reserved.© 2005 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
AppendixAppendix
RealTimeStylus Plug-in InterfacesRealTimeStylus Plug-in Interfaces
RealTimeStylusRealTimeStylus
1.1. PerformancePerformanceDe-couples rendering and collection of inkDe-couples rendering and collection of ink
2.2. FlexibilityFlexibilityYou can control exactly how and when the You can control exactly how and when the ink is collected and renderedink is collected and rendered
3.3. ExtensibleExtensibleYou can create your own plug-ins to render You can create your own plug-ins to render and collect inkand collect ink
RealTimeStylusRealTimeStylus
1.1. PerformancePerformanceGives you the opportunity to ensure users Gives you the opportunity to ensure users see no lag when creating inksee no lag when creating ink
Collection of stylus input is moved off the Collection of stylus input is moved off the UI thread and is collected asynchronouslyUI thread and is collected asynchronously
Provides direct access to stylus input Provides direct access to stylus input eveneven before it is renderedbefore it is rendered
Queued events allow for separation of Real Queued events allow for separation of Real Time Collection thread and UI threadTime Collection thread and UI thread
Minimizes thread usage Minimizes thread usage (From 5 per instance to 1 (From 5 per instance to 1 per process)per process)
RealTimeStylusRealTimeStylusFocus:Focus:•No lag renderingNo lag rendering
RealTimeStylusRealTimeStylus
2.2. FlexibilityFlexibilityPlug-in based architecturePlug-in based architecture
You determine how you render and collect You determine how you render and collect ink based on which plug-ins you chooseink based on which plug-ins you choose
Plug-ins can be dynamically added, Plug-ins can be dynamically added, removed or re-orderedremoved or re-ordered
RealTimeStylusRealTimeStylus
3.3. ExtensibleExtensibleAllows for the creation of new Real Time Allows for the creation of new Real Time componentscomponents
Custom dynamic rendering, gesture Custom dynamic rendering, gesture recognition, disabled regions, etc.recognition, disabled regions, etc.
Customize your ink collection and Customize your ink collection and rendering to suit your needrendering to suit your need
Architecture (Overview)Architecture (Overview)
Pen Service
RealTimePlugIn(IStylusSyncPlugin)
RealTimeStylus
queue
RealTimePlugIn(IStylusSyncPlugin)
“Inking” Thread UI Thread
Ink CollectingObject
(IStylusAsyncPlugin)
storage
InkCollector
RealTimeStylus Plug-insRealTimeStylus Plug-ins
Object that implements a synchronous or Object that implements a synchronous or asynchronous interfaceasynchronous interface
SyncPlugin: fast custom renderingSyncPlugin: fast custom renderingAsyncPlugin: ink data collectionAsyncPlugin: ink data collection
Can manipulate packet dataCan manipulate packet dataRead, Modify, Add, or IgnoreRead, Modify, Add, or Ignore
Tablet OS includes two plug-insTablet OS includes two plug-insDynamic RenderingDynamic RenderingGesture Recognition Gesture Recognition
SDK includes two sample plug-insSDK includes two sample plug-insCustom RenderingCustom RenderingConstrained RegionConstrained Region
RealTimeStylus PluginRealTimeStylus PluginFocus:Focus:•Flexibility in rendering ink Flexibility in rendering ink •Flexibility in collecting stroke Flexibility in collecting stroke datadata
Journal Note ReaderJournal Note Reader
Journal Note ReaderJournal Note Reader
Provides Read-only access to Journal Provides Read-only access to Journal NotesNotes
Exposes all attributes in Journal filesExposes all attributes in Journal filesInk, Images, Title, Stationary, etc.Ink, Images, Title, Stationary, etc.
Enables your application to consume common Enables your application to consume common note filesnote files
JNT file is presented as an XML JNT file is presented as an XML documentdocument
Native and Managed versionsNative and Managed versions
DDownloadownload as an SDK update as an SDK update
Journal Reader with Ink Journal Reader with Ink DividerDividerFocus:Focus:•Easy to consume Journal filesEasy to consume Journal files