Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 214 times |
Download: | 1 times |
Lin-Kai Chen VClab
Content Adaptation Framework:Bringing the Internet to Information
Appliances
Content-Based Transcoding of Images
in the Internet
IBM T.J Watson Research Center
J.R Smith,R Mohan and Chung-Sheng Li
Lin-Kai Chen VClab
Introduction
• This paper present Content Adaptation Framework for customizing internet content for diverse client devices.
• Converting content items into alternate modalities and resolutions so as to meet the characteristics of the client devices.
• Different modalities: video,image,text,audio.• Different fidelities: reduced resolution and
summaries.
Lin-Kai Chen VClab
Internet
Universal Access
Via
Content
Adaptation
Information
ContentIn the
office
On the road
Lin-Kai Chen VClab
Introduction(Cont.)
• The content can be adapted on the fly or transcoded in advance.
• This content adaptation can be performed at server client or proxy.
Lin-Kai Chen VClab
Content adaptation Framework(Cont.)
• The components are:• Analysis: To extract meta-data, such as resourc
e requirements and the type and purpose of the content, used in guiding subsequent transcoding and selection process.
• Transcoding: Based on the capabilities of client devices,different transcoding modules are employed to generate versions of the content in different resolutions and modalities.
Lin-Kai Chen VClab
Content adaptation Framework(Cont.)
• Selection:Selects the resolutions or modalities that best meet the client capabilities.
• Rendering:The selected content is then rendered in a suitable delivery format(eg.HTML).
• Description:The InfoPyramid is used to represent the multiple resolutions and modalities of the transcoded content.
Lin-Kai Chen VClab
Content Analysis
• Resource requirement– Content size in bits.
– Display size:height,width,and area.
– Streaming bit-rate.
– Color requirements.
– Software requirements:support for specific codecs.
– Hardware requirements.
Lin-Kai Chen VClab
Content Analysis
• Content type and purpose– The type of content decides which specific transcoding
algorithm are applied.
– One focus of the content analysis is images as they currently account for the largest amount of web traffic in terms of bits.
– Analyzing images to determine their type and purpose.
Lin-Kai Chen VClab
Image content analysis
Image type
Decision tree
Features
Training set
Parameters
Real time trans.
X[m.n]
Xr, Xg, Xb
μsPv, Vv μh, μv
Color B/W Gray
B/W graphicphoto
Purpose class
Related text
Lin-Kai Chen VClab
Image Content Analysis
• Image type classes:– BWG : b/w graphic.
– BWP : b/w photo.
– GRG : gray graphic.
– GRP : gray photo
– SCG : simple color graphic
– CCG : complex color graphic
– CP : color photo.
Lin-Kai Chen VClab
Image Content Analysis
• Image purpose classes– ADV: advertisement, ie. Banner ads.
– DEC: decoration, ie. Background textures.
– BUL: bullets, points,dots.
– RUL: rules,lines,separators.
– MAP: maps, ie,images with click focus.
– INF: information:icons,logos.
– NAV: navigation, ie,arrows
– CON: content related: news photos.
Lin-Kai Chen VClab
Image Content Analysis
• Image type classification– Decision tree as a classifier.
– It consists of 5 decision points, each of which utilizes a set of features extracted from the images.
– Classification parameters from the training set of 1282 images.
Lin-Kai Chen VClab
Color vs. non-color
• Each image X[m,n] has xr, xg, xb.
• Using the measurement of the mean saturation per pixel μs. The saturation channel ys .
ys = max(xr, xg, xb) – min(xr, xg, xb)
• μs discriminates well between color and non-color images.
nm
ss nmyMN ,
],[1
Lin-Kai Chen VClab
B/W vs. Gray
• Intensity channel
• Intensity entropy
• ,where
•
bgrv xxxy 106030 ...
][log][ kpkpPkv 2
255
0
nm
v
otherwise
nmykif
MNkp
,
],[][
0
11
Lin-Kai Chen VClab
BWG vs. BWP
• It uses the minimum of the mean number of intensity switches in horizontal and vertical scans of images.
Lin-Kai Chen VClab
GRG vs. GRP
• Using the intensity switch measure Wv and intensity entropy Pv .
• As to switch measure, it has a much lower value for b/w graphics.
Lin-Kai Chen VClab
SCG vs. CCG vs. CP
• The images are transformed to HSV and vector quantized.
• The process generates a 166-HSV color representation of the image y166 ,where each pixel refers to an index in the HSV color look-up table.
• The mean color switch per pixel Wv
• Color entropy P166
Lin-Kai Chen VClab
Decision tree : correct classification rate(%)
Decision1
non-colorvs.
Color
Decision2B/W
vs.Gray
Decision3B/W
Graphicvs.
Photo
Decision4Gray
Graphicvs.
Photo
Decision5Color
Graphic
vs.Photo
88.2 95.3 97.8 77.5 68.2
Image type class correct classification rate(%)
B/Wgraphic
B/Wphoto
Graygraphi
c
Grayphoto
SCG CCG Colorphoto
91 91 65 78 52 42 70
Lin-Kai Chen VClab
Image purpose classification
• Image purpose is detected by applying a rule induction engine on the image type,document context and related text.
• Context for images.– BAK: background, ie <body background=…..>
– INL: inline,ie <img src=…>
– ISM: is map
– REF: referenced , ie <a href=…>
– LIN: linked, ie <a href=….><img src=…></a>
Lin-Kai Chen VClab
Image purpose classification
• Text terms:extracted from the ‘alt’ tag, the image URL address strings,and the text nearby the images in the web documents.
• D={‘ad’,’texture’,’bullet’,’map’,’logo’,’icon’}
Lin-Kai Chen VClab
Transcoding policy
• Transcoding functions– Size:minify,crop,and subsample.
– Fidelity:JPEG,GIF compress, quantize, reduce resolution, smooth,sharpen…etc.
– Color content: reduce color, convert to gray,b/w.
– Substitution: attributes, text,type,purpose, and remove image.
Lin-Kai Chen VClab
Transcoding policy
• For compression, JPEG works well for gray photo graphics but not for graphics.For GIF, the reverse is true.
• The transcoding policies also make use of the image purpose analysis.
Lin-Kai Chen VClab
Conclusion
• The content adaptation can be performed at various nodes in the internet:the server,proxy and client.
• They proposed the content discription scheme based on the InfoPyramid, to the MPEG-7 and W3C standards bodies.
• The system utilizes transcoding policies based on the content classes to transcode and adapt images.