Select. Collect. Reflect.

Bookmarklets as

Applications

Fluent 2012

Gary Flake, CEO / Founder

flake@clipboard.com

Select. Collect. Reflect.

Bookmarklet Demo

Select. Collect. Reflect.

How it works

• Check login state

• Rewrites the page

• Interactive UI

• Analyzes DOM & CSS

• Preview UI

• Transmits Data

• Reports back status

What does our bookmarklet do?

• Fast hit detection

• Robustly render UI

• Secure API calls

Some obvious challenges

elementFromPoint()

CSS sandboxing

XDM + USTORE + HMAC

• Fast hit detection

• Robustly render UI

• Secure API calls

Some obvious challenges

elementFromPoint()

CSS sandboxing

XDM + USTORE + HMAC

1. Get positions and sizes of all possible hits.

2. On mouse hover, look for the “best” hit.

3. Factor in z-index, nesting, and sizes to

disambiguate.

Many problems: slow, imprecise, breaks on reflow.

Safe but naïve hit detection method

• document.elementFromPoint()

• Does everything that we want.

• Will return an iframe if anywhere on an iframe.

• Can use the iframe’s document to find

elements nested (e.g., gmail).

Much better hit detection

• Fast hit detection

• Robustly render UI

• Secure API calls

Some obvious challenges

elementFromPoint()

CSS sandboxing

XDM + USTORE + HMAC

• Programmatically reset

(block, inline, input, li, table, tbody, tr, td)

• Use specific selectors and !important

(avoids having to write brittle CSS)

• Customize jQuery internals

(.style takes flag for important for animations)

Three steps

shared: { 'background-image': 'none', 'background-origin': 'padding-box', 'background-size': 'auto', 'border-spacing': '0', 'border': '0 solid black', 'border-image': 'none', 'bottom':

'auto', 'clear': 'none', 'content': 'normal', 'crop': 'auto', 'cursor': 'auto', 'direction': 'ltr', 'float': 'none', 'font-size-adjust': 'none', 'font-stretch': 'normal', 'font-style': 'normal', 'font-

variant': 'normal', 'font-weight': 'normal', 'height': 'auto', 'left': 'auto', 'letter-spacing': 'normal', 'line-break': 'auto', 'line-height': 'normal', 'margin-bottom': '0', 'margin-left': '0',

'margin-right': '0', 'margin-top': '0', 'max-height': 'none', 'max-width': 'none', 'min-height': 'none', 'min-width': 'none', 'outline-color': 'invert', 'outline-style': 'none', 'outline-width':

'medium', 'overflow-wrap': 'normal', 'padding-bottom': '0', 'padding-left': '0', 'padding-right': '0', 'padding-top': '0', 'position': 'static', 'right': 'auto', 'text-autospace': 'none', 'text-

decoration': 'none', 'text-outline': 'none', 'text-overflow': 'clip', 'text-shadow': 'none', 'text-transform': 'none', 'text-wrap': 'none', 'top': 'auto', 'unicode-bidi': 'normal', 'visibility':

'visible', 'white-space': 'normal', 'width': 'auto', 'word-break': 'normal', 'word-spacing': 'normal', 'z-index': 'auto'

},

block: { 'display': 'block', 'overflow': 'visible', 'overflow-clip': 'auto', 'overflow-style': 'auto', 'overflow-x': 'visible', 'overflow-y': 'visible', 'text-align': 'left', 'text-indent': '0', 'widows':

'2’

},

inline: { 'display': 'inline', 'vertical-align': 'baseline’

},

table: { 'border-collapse': 'separate', 'table-layout': 'auto', 'display': 'table’

},

tableCell: { 'empty-cells': 'show', 'vertical-align': 'baseline', 'display': 'table-cell’

},

tableRow: { 'display': 'table-row’

},

list: { 'list-style-image': 'none', 'list-style-position': 'outside', 'list-style-type': 'disc’

},

link: { 'cursor': 'pointer'

},

listItem: { 'display': 'list-item'

},

textInput: { 'cursor': 'text'

}

Forced CSS resets

Original Site

Not perfect but good enough

Our UI

Their CSS

Inline resets

Our CSSwith

!important

Our Custom jQuery with!important

• Fast hit detection

• Robustly render UI

• Secure API calls

Some obvious challenges

elementFromPoint()

CSS sandboxing

XDM + USTORE + HMAC

• Our code is running on a 3rd party page.

• We don’t want host page to be able to infer

user’s secret.

• We don’t want our API calls to be spoofed.

API challenges

How we send a signed XDP

foo.com/page

XDP.jsclipboard.com

User’s browser Secret Key

Clip data viaeasyXDM.js

HMAC withsecret key

Client local storage in the clipboard.com domain

Promise token for asynch result check

Select. Collect. Reflect.

Doing it yourself

1. Include our embed script

2. Add a DIV

3. ????

4. Profit!!!

Full details at: http://clipboard.com/tools

Add clipping to your page

<!DOCTYPE html>

<html>

<head>

<title>Embed test</title>

<script type="text/javascript"

src="//clipboard.com/js/bookmarklet_boot.js?origin=embed" />

</head>

<body>

<p id="embedMe">Hello World</p>

<div class="clipboardEmbedButton" data-start="#embedMe" />

</body>

</html>

Example

data-start – CSS selector for first (or only) element to be copied.

data-end – optional selector to specify range.

data-theme – light or dark (default)

style – restricted styles (position, display, float, margin, top, right, bottom, left)

Button attributes

Select. Collect. Reflect.

Closing Remarks

• Bookmarklets are one of the few methods for

creating in-browser mashups between services.

• There are a ton a gotchas around security,

safety, and performance.

• See http://clipboard.com/tools for embed

instructions and details.

Bookmarklets as applications

Select. Collect. Reflect.

Thanks!

Select

CollectReflect

Product Basics

Save any part of

any Web page

Organize,

share, and publish

Discover, search,

and transact

Architecture

riak-01

riak-02

riak-03riak-04

riak-05

web-01Node.js + Nginx

web-02Node.js + Nginx

web-03Node.js + Nginx

cache-01

cache-02

cache-03

redis-01

redis-02

thumb-01 thumb-02 job-02

admin-01

job-01