Getting touchy - an introduction to touch and pointer events (Workshop) / JavaScript Days / Munich /...

transcript

getting touchy AN INTRODUCTION TO TOUCH AND POINTER EVENTS

Patrick H. Lauke / JavaScript Days / München / 7 März 2014

1. introduction

2. touch events

3. break

4. pointer events

5. debugging/testing

6. conclusion

patrickhlauke.github.io/touch

“how can I make my website work on touch devices?”

you don't need touch eventsbrowsers emulate regular mouse events

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

(mouseenter) > mouseover > mousemove* > mousedown > (focus) > mouseup > click

*only a single “sacrificial” event is fired

on first tap(mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click

subsequent tapsmousemove > mousedown > mouseup > click

tapping awaymouseout > (blur)

focus/blur only on focusable elements in Opera Mobile and Firefoxmouseout not on iOS Safari and embedded WebView (e.g. iOS Chrome)

emulation works,but is limiting/problematic

1. delayed event dispatch2. mouse-specific interfaces3. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

steamcommunity.com/id/redux_splintered/screenshots/?appid=238260

vimeo.com/ondemand

patrickhlauke.github.io/touch/css-dropdown/mouse-only.html

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2(bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)

“we need to go deeper...”

touch eventsintroduced in iOS 2.0, adopted in Chrome/Firefox/Opera

www.w3.org/TR/touch-events

touchstarttouchmove

touchend

touchcanceltouchentertouchleave

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.htmlBug 128534 - 'mouseenter' mouse compat event not fired when listeners for touch events

touchstart > [touchmove]+ > touchend > (mouseenter) > mouseover >

mousemove > mousedown > (focus) > mouseup > click

(mouse events only fired for single-finger tap)

on first taptouchstart > [touchmove]+ > touchend >

(mouseenter) > mouseover > mousemove > mousedown > (focus) > mouseup > click

subsequent tapstouchstart > [touchmove]+ > touchend > mousemove > mousedown > mouseup > click

tapping awaymouseout > (mouseleave) > (blur)

too many touchmove events abort the tap (after touchend)too many touchmove events on activatable elements lead to touchcancel

not all browsers consistently send the touchmove

some browsers outright weird...

Browser/Android 4.3(AppleWebKit/534.30)

mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click

Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)

touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click

touch eventsvs

limitations/problems

why the delay?double-tap to zoom

(mostly anyway)

what if browsers didn't wait?

Puffin/Android (popular in Korea?) puffinbrowser.comdouble-tap zooms and fires mouse events + click

(also, doesn't support touch events at all)

UCWeb/Android (popular in China?) ucweb.comno delay...but only because it doesn't implement double-tap to zoom

when does the delay happen?

patrickhlauke.github.io/touch/tests/event-listener.html

“how can we make it feel responsive like a native app?”

react to events fired before the 300ms delay...

touchstartfor an “immediate” control

touchendfor a control that fires after finger lifted

interlude:simple feature detection for

touch events

if ('ontouchstart' in window) {

/* some clever stuff here */

/* older browsers have flaky support so more hacky tests needed... use Modernizr.touch or similar */

/* common performance “trick” */

var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click');

blah.addEventListener(clickEvent,function() { ... }, false);

don't make it touch-exclusive

hacks.mozilla.org/2013/04/detecting-touch...

github.com/Modernizr/Modernizr/issues/548

if ('ontouchstart' in window) {

/* browser supports touch events but user is not necessarily using touch (exclusively) */

hybrid devicestouch + mouse + keyboard

patrickhlauke.github.io/touch/tests/event-listener_show-delay-naive-event-fix.html

bugzilla.mozilla.org/show_bug.cgi?id=888304

Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown >

(focus) > mouseup > click

Blackberry PlayBook 2.0 + mouse – like desktop mousemouseover > mousedown > mousemove > mouseup > click

Android + keyboard – like desktop keyboardfocus > click

VoiceOver enables keyboard access on iOS

iOS + keyboard – similar to touchfocus > touchstart > touchend >

(mouseenter) > mouseover > mousemove > mousedownblur > mouseup > click

iOS + VoiceOver – similar to touchfocus > touchstart > touchend >

(mouseenter) > mouseover > mousemove > mousedownblur > mouseup > click

Android + TalkBack – keyboard/mouse hybridfocus > blur > mousedown > mouseup > click > focus(?)

(though seems a bit flakey...)

touch or mouse or keyboard

touch and mouse and keyboard

/* doubled-up event listeners */

foo.addEventListener('touchend', someFunction, false);

foo.addEventListener('click', someFunction, false);

foo.addEventListener('touchstart',function(e) {/* prevent delay + mouse events */e.preventDefault();

}, false);

/* doubled-up event listeners */

foo.addEventListener('touchend', someFunction, false);

foo.addEventListener('click', someFunction, false);

preventDefaultcan kill scrolling, pinch/zoom, etc on

touchstart/touchmove

github.com/ftlabs/fastclick

browsers working to remove delay when possible

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Bug 922896 - Optimizations to remove 300ms touch > mouse events delay[RESOLVED - FIXED]

Issue 169642: Remove ~300ms delay on tap for mobile sites [...]

[...] no more 300ms click delay, or double-tap zoom, on mobile websites

Chrome 32 for Android removes delay also when

updates.html5rocks.com/2013/12/300ms-tap-delay-gone-away

iOS/Safari designed themselves into a cornerwith “double-tap to scroll”

bugs.webkit.org/show_bug.cgi?id=122212

steamcommunity.com/id/redux_splintered/screenshots/?appid=238260

vimeo.com/ondemand

no isolated hover (or focus) on touch devices*

* iOS fakes it, Samsung Galaxy S4 + stylus, ...

http://developer.apple.com/library/IOS/...

Modernizr issue 869: Detecting a mouse user

www.stucox.com/blog/you-cant-detect-a-touchscreen

avoid hover-based interfaces?

complement for keyboard / touch!

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html

patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html

patrickhlauke.github.io/touch/particle/2(bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)

touchstart > [touchmove]+ > touchend >(mouseenter) > mouseover >

mousemove* > mousedown > (focus) >mouseup > click

*mouse event emulation fires only a single mousemove

doubling up handling of mousemove and touchmove

var posX, posY;

function positionHandler(e) {posX = e.clientX;posY = e.clientY;

canvas.addEventListener('mousemove', positionHandler, false);

var posX, posY;

function positionHandler(e) {/* handle both mouse and touch? */

canvas.addEventListener('mousemove',positionHandler, false);

canvas.addEventListener('touchmove',positionHandler, false);

interface MouseEvent : UIEvent {readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute unsigned short button;readonly attribute EventTarget relatedTarget;void initMouseEvent(...);

www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent

interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;

www.w3.org/TR/touch-events/#touchevent-interface

interface Touch {readonly attribute long identifier;readonly attribute EventTarget target;readonly attribute long screenX;readonly attribute long screenY;readonly attribute long clientX;readonly attribute long clientY;readonly attribute long pageX;readonly attribute long pageY;

www.w3.org/TR/touch-events/#touch-interface

patrickhlauke.github.io/touch/touchlist-objects

var posX, posY;

function positionHandler(e) {if ((e.clientX)&&(e.clientY)) {

posX = e.clientX;posY = e.clientY;

} else if (e.targetTouches) {posX = e.targetTouches[0].clientX;posY = e.targetTouches[0].clientY;e.preventDefault();

canvas.addEventListener('mousemove',positionHandler, false );

canvas.addEventListener('touchmove',positionHandler, false );

www.splintered.co.uk/experiments/archives/paranoid_0.5

tracking finger movementover time ... swipe gestures

patrickhlauke.github.io/touch/swipe

patrickhlauke.github.io/touch/picture-slider

don't forget mouse/keyboard !

bradfrostweb.com/demo/mobile-first

touchmove fires...a lot!

do absolute minimum on each touchmove

(usually: store new coordinates)

heavy JavaScript on requestAnimationFrame

setInterval

patrickhlauke.github.io/touch/touch-limit

why stop at a single point?multitouch support

interface TouchEvent : UIEvent {readonly attribute TouchList touches;readonly attribute TouchList targetTouches;readonly attribute TouchList changedTouches;readonly attribute boolean altKey;readonly attribute boolean metaKey;readonly attribute boolean ctrlKey;readonly attribute boolean shiftKey;

www.w3.org/TR/touch-events/#touchevent-interface

for (i=0; i<e.targetTouches.length; i++) {

posX = e.targetTouches[i].clientX;posY = e.targetTouches[i].clientY;

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

iOS/iPad even preventDefault()can't override 4-finger gestures

iOS7/Safari even preventDefault()can't override back/forward swipe gestures

multitouch gestures

/* iOS/Safari has gesture events for size/rotation, not supported in Chrome/Firefox/Opera, not part of the W3C Touch Events spec. */

gesturestart, gesturechange, gestureend

e.scale, e.rotation

patrickhlauke.github.io/touch/iosgestures

patrickhlauke.github.io/touch/iosgestures/image.html

/* with some trigonometry we can replicate these from basic principles. */

var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);

patrickhlauke.github.io/touch/pinch-zoom-rotate

patrickhlauke.github.io/touch/picture-organiser

older/cheaper devices/OS versionsand multitouch?

HTC Hero – Android 2.1

LG Optimus 2X – Android 2.3.7

ZTE Open – FirefoxOS 1.1

touch events andInternet Explorer...

blogs.msdn.com/...

www.w3.org/TR/pointerevents

not just some “not invented here” new approach for IE10+

html5labs.interoperabilitybridges.com/prototypes/...

code.google.com/p/chromium/issues/detail?id=162757

bugzilla.mozilla.org/show_bug.cgi?id=822898

...what about Apple?

bugs.webkit.org/show_bug.cgi?id=105463RESOLVED WONTFIX?!

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

mousemove* >pointerover > mouseover >

pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove >

gotpointercapture > focus >

pointerup > mouseup >lostpointercapture >

pointerout > mouseout >pointerleave > mouseleave >

mouse events fired “inline” with pointer events(for a primary pointer, e.g. first finger on screen)

vendor-prefixed in IE10MSPointerDown etc

navigator.msMaxTouchPoints-ms-touch-action

unprefixed in IE11

/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */

interface PointerEvent : MouseEvent {readonly attribute long pointerId;readonly attribute long width;readonly attribute long height;readonly attribute float pressure;readonly attribute long tiltX;readonly attribute long tiltY;readonly attribute DOMString pointerType;readonly attribute boolean isPrimary;

www.w3.org/TR/pointerevents/#pointerevent-interface

simple feature detection for pointer events

if (window.PointerEvent) {

/* some clever stuff here but this covers touch, stylus, mouse, etc */

/* still listen to click for keyboard! */

if (navigator.maxTouchPoints > 1) {

/* multitouch-capable device */

patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html

no need for separatemouse or touchevent listeners

/* touch events: separate handling */

foo.addEventListener('touchmove', ... , false);foo.addEventListener('mousemove', ... , false);

/* pointer events: single listener for mouse, stylus, touch */

foo.addEventListener('pointermove', ... , false);

no need for separatemouse or touch

code to get x / y coords

/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */

interface PointerEvent : MouseEvent {readonly attribute long pointerId;readonly attribute long width;readonly attribute long height;readonly attribute float pressure;readonly attribute long tiltX;readonly attribute long tiltY;readonly attribute DOMString pointerType;readonly attribute boolean isPrimary;

www.w3.org/TR/pointerevents/#pointerevent-interface

but you can distinguishmouse or touch or stylus

if needed

foo.addEventListener('pointermove', function(e) { ... switch(e.pointerType) { case 'mouse': ... break; case 'pen': ... break; case 'touch': ... break; default: /* future-proof */ ... } , false);

pointer eventsvs

limitations/problemsof mouse event emulation

“how can we make it feel responsive like a native app?”

you can preventDefault()most mouse compatibility events on pointerdown

...but click is not considered mouse compatibility event, and the 300ms delay is not affected by this.

touch-action: auto|none|[pan-x][pan-y]

www.w3.org/TR/pointerevents/#the-touch-action-css-property

only prevents default touch action (e.g. double-tap to zoom)does not stop synthetic mouse events nor click

touch-action:none

patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html

mousemove >pointerover > mouseover >

pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove >

gotpointercapture > focus >

pointerup > mouseup >click >

lostpointercapture > pointerout > mouseout >pointerleave > mouseleave

touch-action:nonekills scrolling, long-press,

pinch/zoom

msdn.microsoft.com/en-us/library/ie/dn265029(v=vs.85).aspx

msdn.microsoft.com/en-us/library/ie/jj152135(v=vs.85).aspx

patrickhlauke.github.io/touch/css-dropdown/mouse-only-aria-haspopup.html

channel9.msdn.com/Blogs/IE/IE11-Using-Hover-Menus-with-Touch

touch-action:none

patrickhlauke.github.io/touch/particle/2a

what about

multitouch?

patrickhlauke.github.io/touch/tracker/mouse-tracker_touch-action-none.html

/* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */

var points = [];

switch (e.type) {

case 'pointerdown': /* add to the array */ break;

case 'pointermove': /* update the relevant array entry's x and y */ break;

case 'pointerup': /* remove the relevant array entry */ break;

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html

/* like iOS/Safari, IE10/Win has higher-level gestures, but these are not part of the W3C Pointer Events spec.

Replicate these from basic principles again? */

pointer events as the future?

what about backwards-compatibility?

touchstart > [touchmove]+ > touchend >[300ms delay] > mouseover > mousemove >

mousedown > mouseup > click

+pointerover > mouseover > pointerdown >

mousedown > pointermove > mousemove > pointerup > mouseup > pointerout > mouseout >

[300ms delay] > click

www.w3.org/community/touchevents

polyfills for pointer events(code for tomorrow, today)

handjs.codeplex.com

www.catuhe.com/msdn/handjs

github.com/Polymer/PointerEvents

www.polymer-project.org

utility libraries(because life is too short to hand-code gesture support etc)

eightmedia.github.io/hammer.js

jQuery Mobile? Sencha Touch? …check for support of IE10+ and “this is a touch device” assumptions

debugging/testing

Issue 181204: Emulate touch events - event order different from real devices

Bug 920956 - DevTools touch emulation: suppress regular mouse events ...

developers.google.com/chrome-developer-tools/docs/console#monitoring_events

developers.google.com/chrome-developer-tools/docs/remote-debugging

Bug 861876 - [...] multiple mousemoves being fired

Bug 861876 - [...] preventDefault on touchstart doesn't stop synthetic mouse events

Getting touchy - an introduction to touch and pointer events (Workshop) / JavaScript Days / Munich /...

Technology