Learning to remove Internet advertisements

transcript

Dec 6, 2004 12004 Michigan Technological University

Nicholas KushmerickNicholas Kushmerick

Department of Computer Science,Department of Computer Science,

University College Dublin, IrelandUniversity College Dublin, Ireland

Learning to remove Internet Learning to remove Internet advertisementsadvertisements

Presented by Bo ZhangDepartment of Computer Science Michigan Technological University

OverviewOverview

BackgroundBackground

Introduction of ADEATERIntroduction of ADEATER

Design of ADEATERDesign of ADEATER

EvaluationEvaluation

Related WorkRelated Work

Conclusion and Future WorkConclusion and Future Work

BackgroundBackground Negative Impact of advertisement images on InternetNegative Impact of advertisement images on Internet

Slow down the speed of browsing Consume resources of computer Extra costs for users

Advertisement Image

Introduction of ADEATERIntroduction of ADEATER

Definition:Definition:

- A browsing assistant that automatically removes advertisement images from Internet pages.

Property:Property:

Rules generated from learning algorithm

Introduction of ADEATERIntroduction of ADEATER ExamplesExamples

Design of ADEATER Design of ADEATER

System ArchitectureSystem Architecture

Design of ADEATERDesign of ADEATER Encoding instanceEncoding instance

Fixed–width feature vector

Images enclosed in anchor tag <A> is a candidate advertisement

Geometric features of an image: -Height <IMG height=90> -Width <IMG width=90> -Aspect ratio (ratio of width to height)

Local feature: -Whether destination URL and image URL are in the same internet

domain www.ee.mtu.edu/page.html www.cs.mtu.edu/image.jpg YES

www.dell.com/notebook.html www.cs.mtu.edu/image.jpg No

Encoding instanceEncoding instance

Caption feature: -Words occuring in enclosing <A> tag with phrase length<K

and phrase count >M -K is maximum phrase length -M is minimum phrase count

Alt Feature -Set of “alternate” words in the <IMG> tag (<IMG alt=“ad”>)

with phrase length<K and phrase count >M -K is maximum phrase length -M is minimum phrase count

Ubase, Udest, Uimg

-Words occuring in base URL, destination URL, image URL with phrase length<K and phrase count >M -K is maximum phrase length -M is minimum phrase count

Stop list -Low-information terms (“http”, “www”, ”jpg”, etc.)

Samples of HTML page

Encoding of samples

Encoding of samples (cont)

Gathering examplesGathering examples

AD samples are generated by ADGRABBER browsing assistant

Identifier candidate advertisements

Generate vector encoding

NON-AD samples are generated by a custom-built Internet spider

Extract images from randomly-generated URLs.

Learning rules

Algorithm - C4.5 decision tree learning algorithm

Properties - Quick on-line execution of classifier - Not be overly sensitive to missing features or noises - Scale well and insensitive to irrelevant features

Examples of rules - If aspect ratio > 4.5833, alt doesn’t contain “to” but does

contain “click+here”,and Udest doesn’t contain “http+www”, then instance is an AD

- If Ubase does not contain “messier”, and Udest contains the “redirect+cgi”, then instance is an AD

Removing advertisementsRemoving advertisements

Process

- Fetch HTML pages from Internet - Identify candidate advertisements - Classify instances with learned rules - Replace the image’s URL with the URL of an inconspicuous low-bandwidth image

Implementation

- Removal module as a proxy server

Evaluation

Speed and accuracySpeed and accuracy

Experiment setting

Total samples - AD: 459 examples

- NON-AD: 2820 examples

10-fold cross-validation - Training set: 90% examples - Test set: 10% examples

Off-line training phase: 5.8 minutes

On-line classification phase: 70 msec/image

Average accuracy: 97.1%

Evaluation Learning curvesLearning curves

Simple methodology - Not recalculate feature set Realistic methodology - Recalculate feature set

Evaluation

Alternative encodingsAlternative encodings

Related Work Muffin: Filtering web pages

ImageKill Filter: Hand-crafted rules

ImageKill.minheight

- Only remove images which are at least n pixels high

ImageKill.minwidth

- Only remove images which are at least n pixels wide

ImageKill.ratio

- Remove images which are more than n times as wide as

they are high

ImageKill.exclude

- Don't remove images that match the given string/regexp

Related Work

WebFilter: Filtering web pages

Solution

- User provides a list of URL templates and corresponding

filter scripts

Related Work

Junkbuster: Junkbuster: Filtering web pages

Solution

- User provides a block file

Related Work

Smokey: Detect abusive messagesSmokey: Detect abusive messages

Solution

- Training samples and generate rules by training - Parse messages and generate feature vector - Classify the feature vector with rules generated

Conclusion and Future Work

ConclusionConclusion

High accuracy

Modest resource cost (processing time, training samples)

Future WorkFuture Work

Incremental learning algorithm

More efficient feature selection mechanism

Thank you!Thank you!

Learning to remove Internet advertisements

Documents