Enhancing the Web With End-User ProgrammingTak Yeon Lee, Ben Bederson
Outline Problem
Inefficiencies on the WebChallenge
Building extension is very hardOur goal
Expressive and Easy Programming Env.
- Improves a wide range of tasks- No programming skills required
Snapshot
ANY WEBSITE
Inefficiencies on the Web
1. Missing relevant information
“I want to see the pictures.”
Detail page“Is it a good value?”
Kelly’s Blue book
“How far is this place?”
Google Map
“Is the model reliable?”
Auto forums
2. Removing unwanted information
AD
AD
2. Removing unwanted information
3. Repetitive Operations
Howe-TheRiseOfCrowdsourcing-2006.pdf
Kittur-CrowdsourcingUserStudies-2008.pdf
Brabham-CrowdsourcingAsAModelFor-2008.pdf
Alonso-CrowdsourcingForRelevanceEvaluation-2008.pdf
Kittur-08-001.pdf
60947(1).pdf
Alonso2008.pdf
10.1.1.154.444.pdf
“I want to download all PDFs named as [first author]-[title]-[year].pdf ”
1. Finding mistakes
2. Add comments3. Calculate the total points4. Submit to grade server
Grading Procedure
Takes 4-8 hoursto grade 70
students
4. Finding a needle in a haystack
MyDouble.zero
MyDouble.zero
MyDouble.zero
Grading with an extensionSummary of Potential Mistakes
1. Click to scroll to the code line of the mistake
2. Click to add predefined comment
3. Setting partial credit
4. Report total score to another server
“Previously it took several hours, but now I can finish in 40 minutes.”
- Lecturer“Three times faster than before, at least!”
-Grading TA
Browser Extensions add new features and personalize experience. Over 50,000 on Chrome Web Store
AdBlock>15 million users
Photo Zoom>2 million users
Print AllE-mails in a folder
Building an extension requires• Programming knowledge (HTML, CSS, and Ajax)• Time and effort of development
• The grading example took >40 hours
OUR GOAL IS TO BUILD
Expressive and Easy programming env.
The toolkit should support tasks like...• Automating repetitive jobs• Integrating data from multiple sources• Custom filtering• Information Summary
We defined a domain language model having
Pick, Change, Add operations as generic functionalities of the extensions.
Pick extract information from web pages or elements.
Pick Change Add model
The rise of crowdsourcing.pdfCrowdsourcing user studies...Crowdsourcing as a model ...
http://sistemas-humano.com...http://dl.acm.org/citation.cfm?id=1357127http://con.sagepub.com/...
<A> tags
URLs
Changemanipulates dataset.
Pick Change Add model
• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...
• String manipulation• concatenate, split, find,
replace, ...
• Filter• Sort
1
2
3
3
3
6
9
[a] [b]
[a] * [b]
Changemanipulates dataset.
Pick Change Add model
• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...
• String manipulation• join, split, find, replace, ...
• Filter• Sort
I
am a
boy
-
I-am a-boy
[a] [b]
Join [a] with [b]
Changemanipulates dataset.
Pick Change Add model
• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...
• String manipulation• join, split, find, replace, ...
• Filter• Sort
a.jpg
b.gif
c.png
png
c.png
[a] [b]
Filter [a] having [b]
Changemanipulates dataset.
Pick Change Add model
• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...
• String manipulation• join, split, find, replace, ...
• Filter• Sort
Toronto
Seoul
Paris
[a]
Sort [a]
Paris
Seoul
Toronto
Addmodify the original web pages.
Pick Change Add model
• Create new elements• image, text, button, ...
• Modify element’s style• Show / Hide, changing color, fonts, ...
• Triggering events• click, ...
• Type text
[1]. Pick ad banner elements[2]. Hide [1]
AD
AD
Pick Change Add model
e.g. Removing AD banners
e.g. Downloading files with custom name pattern
The rise of crowdsourcing.pdf
Crowdsourcing user studies with Mechanical Turk.pdf
Crowdsourcing as a model for.pdf
Crowdsourcing for relevance evaluation.pdf
[3]. Pick download links
[1]. Pick title text
[2]. Merge [1] with name pattern “.pdf”
[4]. Add [2] to ‘download’ attribute of [3]
Pick Change Add model
OUR GOAL IS TO BUILD
Expressive and Easy programming env.
• Requires no programming knowledge
• Programming-by-Example techniques• Telling computer what I want, without
worrying how to do.
?
?
PROGRAM
OUTPUT
INPUT
Traditional Programming Programming-by-Example
int sum=0;for(int v in input) {
sum=sum+v;} return sum;
1, 2, 3
6
!
Programming-by-Example technique
Paris
NewYork
Seoul
Tokyo
.jpg
Paris.jpg
NewYork.jpg
Seoul.jpg
Tokyo.jpg
[1] [2]
Merge [1] with [2]
Programming-by-Example techniquePrograms with multiple inputs
e.g. Adding suffix to strings
Input
Output
Paris, France
Istanbul, Turkey
Seoul, Korea
Osaka, Japan
Korea
Japan
[1] [2]
Seoul, Korea
Tokyo, Japan
2
Filter [1] exist in [2]
[3]
Count [3]
Programming-by-Example techniqueMulti-step programming e.g. Counting items of [1] that exist in [2]
London, UK
LA, US
Beizing, China
Toronto, Canada
UK
Canada
China
[1] [2]
London, UK
Beizing, China
Toronto, Canada
3
[3]
Count [3]
Programming-by-Example techniqueMulti-step programming e.g. Counting items of [1] that exist in [2]
Filter [1] exist in [2]
Demo time
ConclusionA new opportunity for end-users to personalise the web by creating their own extensions.
Programming-by-Example techniques with Pick, Change, Add model
Future work includes• Usability Tune up • Evaluation of the system’s learnability
Please email us if you have an idea for extensionsTak Yeon Lee <[email protected]>