Date post: | 21-Jan-2018 |
Category: |
Technology |
Upload: | dani-gutierrez-porset |
View: | 568 times |
Download: | 0 times |
Multimedia Services:Image
Sep-2015
Dani Gutiérrez PorsetAssociate Professor
Communications Engineering
Eman ta zabal zazu
2 2Multimedia Services:
Image
Thanks, Licences and Tools
● Thanks to people and organizations who took or take part in free software and free knowledge projects, specially Wikimedia Foundation and KDE
● This presentation is licensed as CC BY-SA 3.0 EShttp://creativecommons.org/licenses/by-sa/3.0/es/
● This presentation has been made with KDE, LibreOffice, Inkscape, Gimp, Chromium, Firefox
3 3Multimedia Services:
Image
Sources and References
● Images from Wikimedia Foundation, if not referenced other source. Logos and trademarks belong to respective organizations
● Texts:
– Wikipedia pages and referenced articles and material– “Guide to Voice and Video over IP” - Sun, Mkwawa, Jammeh,
Ifeachor– “Video over IP” - Wes Simpson– “Computer Networking, a top-down approach” - Kurose, Ross
4 4Multimedia Services:
Image
Index
● Introduction● Color● Raster images● Compression● Image files and formats
5 5Multimedia Services:
Image
Visual perception in humans
● Images and colors are sensations in our brain● Photoreceptors in eye:
– Rods, for brightness– LMS cones for color:
● S (short wavelength = blue)● M (medium wavelength = green)● L (long wavelength = red)
Introduction
6 6Multimedia Services:
Image
Sensitivity of human eye/brain
● More sensible to brightness than color● Less sensitive to the higher spatial frequency components● Less sensitive to quantizing distortion at high luminance
levels● Not all theoretical colors can be perceived (“imaginary
colors“)● More sensible to green. Actually, color sensitivity depends
on bright
Introduction
7 7Multimedia Services:
Image
Image Devices andCorrections (1/2)
● Devices:– Input: camera, eye,...– Output: monitor, projector, printer,...
● Gamut: which colors can be perceived or generated by a device
● Corrections:– For Brightness: Gamma: Mathematical function to consider human
perception of brightness. Relates Luma and Luminance– For Color: ICC profile: color space that characterizes colors in input
or output device, and uses a Profile Connection Space
Introduction
8 8Multimedia Services:
Image
Image Devices andCorrections: Workflow (2/2)
● In capturing devices, e.g. camera, when saving an image:– If raw format is used no correction applied– If non-raw format (e.g. JPEG), image is saved with
gamma encoded or compressed
● Each display device has a gamma value to decode gamma compressed images
Introduction
9 9Multimedia Services:
Image
Types of Digital Still Images
● Raster or Bitmap: represented by a matrix of pixels1
● Vector: represented by vectorial elements: points, vectors (strokes or paths: bézier curves), text,...– No quality loss if size or rotation altered.– for 2D and 3D. E.g.
● 2D: SVG, MathML, POV-Ray● 3D: Blender .blend, CAD .dwg,...
● Possible to mix raster and vectorial in some formats, e.g. SVG, WMF, EMF, PDF, EPS, Postscript, SWF (Shockwave Flash, includes animations,...)
1: All of the next slides in this presentation refer to raster images
Introduction
10 10Multimedia Services:
Image
Applications of Raster Images
● Icons: PNG, BMP, SVG● Web: JPG, PNG, GIF. Streamable using
progressive (interlaced) rendering● Cameras: RAW, EXIF● Graphic Edition: formats with lossless
compression, (proprietary formats)● Print Publishing: TIFF, PDF
Introduction
11 11Multimedia Services:
Image
Pixel Resolution
● For Surface processing devices (e.g. camera, display)
● Concept: pixel dimensions or Width x height pixels● Units: pixels (or megapixels,...)● For an image file it's the real size in pixels
Introduction
12 12Multimedia Services:
Image
Standards forPixel Resolution
● CIF (and variations)● SDTV (Standard Definition TV)● HDTV (High Definition TV): 1080p, 1080i, 720p● 4K family● 8K and Ultra HD TV● VGA, XGA,...
http://en.wikipedia.org/wiki/Graphics_display_resolution https://en.wikipedia.org/wiki/Common_Intermediate_Format https://en.wikipedia.org/wiki/List_of_common_resolutions
Introduction
13 13Multimedia Services:
Image
Aspect ratio
● Ratio of width to height pixels expressed as A:B● Some frequently used e.g. 4:3, 16:9 (HDTV), 21:9● Types:
– SAR: Storage Aspect Ratio– DAR: Aspect Ratio for the Display– PAR: Pixel Aspect Ratio. PAR = DAR / SAR
● Scaling, cropping and vertical/horizontal bands are used to adapt SAR to DAR
Introduction
14 14Multimedia Services:
Image
Spatial Resolution
● For line processing devices (e.g. scanner and printer)
● Concept: granularity in a line● Units: ppi (pixels per inch) or dpi (dots per inch)● For an image file the number of dpi or the real size
in metric units are just metadata
Introduction
15 15Multimedia Services:
Image
Introduction to Color
● Color spectrum (increasing wavelengths):
● Cases:– Device independent models– Device/eye dependent models
Color
16 16Multimedia Services:
Image
Color models and Color spaces
● Color model: mathematical model that uses tuples to reference colors– e.g. RGB, CMYK, YUV, HSV, LAB,...– Conversions between models with matrices
● Color space: organization of colors to represent colors precisely in terms of perception, coming:– From a model transformation for a color model, e.g.
RGB: sRGB, Adobe RGB,...– From a list of colors, e.g. Pantone System
● Profile Connection Space: reference color space used for conversions : CIELAB and CIEXYZ
● Terms “Model” and “Space” sometimes mixed
Color
17 17Multimedia Services:
Image
RGB Color model
● Red, Green, Blue. RGBA: 4th component A = Alpha● Additive model● Device dependent● Uses: electronic displays● Cube● Problems:
– Not efficient with “real-world” images– Not efficient for some image processing, e.g. modify intensity
Color
18 18Multimedia Services:
Image
CMY, CMYK Color model
● Cyan, Magenta, Yellow● Subtractive model● Device dependent● Derived from old RYB● Uses: printing processes, with BlacK
Color
19 19Multimedia Services:
Image
Y'UV, YUV, YCbCr, YPbPr,.Color models
● Components:– Non-color info (black, gray, white info). Terms:
● Luminance: measured. Symbol: Y● Luma: perceived. Symbol: Y'
– Chroma or Chrominance. Symbol: C. Similar concepts:● U, V: color information● Pb, Pr: Chroma for blue (B − Y) and for red (R − Y). For analog encoding (analog component video)● Cb, Cr: Idem for digital encodings● I, Q: In-phase, Quadrature, from rotation of U, V
– Sometimes YUVA: 4th component A = Alpha
● Uses: video systems (Y'UV in PAL, SECAM, YIQ in NTSC)● Advantages:
– Compatible with black and white analog TV– Some Chroma info can be discarded (better perception than RGB with higher compression)
Color
20 20Multimedia Services:
Image
HSV, HSL, HSB, HSIColor models
● Components:– Hue: distinguishes wich colour is– Saturation: intensity or colorfulness– 3rd component: Value, Lightness
(from black to white), Brightnessand/or Intensity
● Cylindrical coordinates. Same RGBcolor model represented in other way
● Uses: by digital artists. More intuitive representation for attributes recognized by human vision
Color
21 21Multimedia Services:
Image
LAB Color model
● LAB or La*b*:– L: Lightness– A, B: Chromacity, in color-opponent dimensions
● Device independent● Designed to approximate human vision● Wider gamut than RGB and CMYK and human eye. More data per
pixel required● Uses:
– Conversions from RGB to CMYK– Interchange, device-independent format
Color
22 22Multimedia Services:
Image
Color depth
● Number of bits to indicate the color:– For a single pixel: Bits per pixel– For each color component: “Bits per channel”, “Bits per color” (RGB,
YCbCr, CMYK) or “Bits per sample”. Alpha channel if transparency is used
● Representations:– Indexed or palette– Direct color:
● Number of bits depending on color space: 8, 15, 16, “True” color = 24, “Deep” color (30, 36, 48) bits. e.g. RGBA 32 bits
● Type e.g. floating values for HDR
Color
23 23Multimedia Services:
Image
Indexed Colors
● Better for images with low number of colors or large areas with solid colors, e.g. cartoons
● Types of palette:– Adaptive: optimized for each image, contains the most
frequent colors in image– Master: contains a miniaturized version of full RGB colors– ...
● Dithering algorithms are usually needed to transform from Direct (e.g. RGB) to Palette schema
Color
24 24Multimedia Services:
Image
Lossless compression methodsCompression
RLE or Run-lengthencoding
Adaptive dictionary
Deflation = combinationof Lz77 and Huffman
Chain codes
TIFF, BMP,...
LZW GIF, TIFF
PNG, TIFF
Monochrome images
25 25Multimedia Services:
Image
Lossy Image compressionand Human Perception
http://en.wikipedia.org/wiki/Image_compression
To relative differences betweendarker tones than lighter tones
To brightness informationthan color information
To green than blue and red
Gamma correctionto convert luminance in luma
Chroma subsampling
Luma with more G info than R, B:Y, Cb, Cr
To spatial low frequencytransitions than high freq
Quantization matrixfor DCT
Human eye is more sensitive... Method to apply:
Compression
26 26Multimedia Services:
Image
YCbCr and Chroma sub-sampling
● Lossy compression method based on Y'CbCr that takes less chroma samples than luma samples (horizontally and/or vertically), based on human visual perception
● Taking a region of w x 2 pixels, the notation is w:h:v– w: pixel width of the region. Generally w=4– h: number of Cb or number of Cr samples in the 1st row of the region– v: number of chroma changes samples between 1st and 2nd row of the region– Exception: not valid for 4:1:0 (region of 4 pixels height)
● Internal packaging in a file or stream: distinct dispositions, e.g. in Y'UV420p (p=“Planar”) first one component for all pixels, second other component for all pixels,...
http://en.wikipedia.org/wiki/Chroma_subsampling
http://en.wikipedia.org/wiki/YUV#Y.27UV420p_.28and_Y.27V12_or_YV12.29_to_RGB888_conversion
Compression
28 28Multimedia Services:
Image
Other Lossy compression methods
● Palette and color quantization for reduced color space (GIF, PNG)
● Transform coding, ej. DCT (JPEG), Wavelet (JPEG 2000, DjVu)
● Fractal compression
Compression
29 29Multimedia Services:
Image
Structure of Image files
● Structure:– File metadata, e.g. Pixel resolution. If chroma subsampling there
maybe distinct pixel blocks– Possible file chunks and chunk metadata– Info for each pixel, packed in pixel by pixel schema or in channel
(planes) schema:● Color: finite or ~ infinite number of colors. Special cases: BW, gray● Alpha channel: Transparency (0%) - Opacity (100%)
● Image File size = f (chunks, resolution, color, compression)http://en.wikipedia.org/wiki/Image_file_format
http://en.wikipedia.org/wiki/Comparison_of_image_file_formats
Image files and formats
30 30Multimedia Services:
Image
Image Formats for Internet
● Supported natively in browsers:– Mandatory: GIF, JPEG, PNG– Some browsers: SVG, PDF,...
● Interlacing: incremental decoding in browsers, so they render progressively images showing first a degraded version that changes to final image:– Better for human perception. Targeted mainly to slower communications– Formats:
● GIF or PNG: images saved with "interlaced" option● JPEG: images saved with "progressive" option
Image files and formats
31 31Multimedia Services:
Image
JPEG compression and file format
● Theorically available compression types:– Lossy compression in three phases (see appendix)– Lossless coding, but not supported in most products
● Modes:– Baseline– Progressive: Data is compressed in multiple passes of progressively
higher detail. Progressive rendering in browsers
● Can embed an ICC color profile● Does not support transparency nor animation● Variations: JPEG 2000 (wavelets compression), JPEG XR,...
Image files and formats
32 32Multimedia Services:
Image
JPEG Encoding flow (1/3):Chroma subsampling
● 1.a Convert from RGB to Y'CbCr● 1.b Apply chroma subsampling, from 4:4:4 to
typically 4:2:2 or 4:2:0 [1st compression, lossy]
Image files and formats
33 33Multimedia Services:
Image
JPEG Encoding flow (2/3):Discard high-frequency variations
● 2.a Split the result image in blocks of 8x8 pixels for luma. MCU (Minimum Coded Unit) or macroblocks = blocks for luma and chroma: 8x8 (if 4:4:4), 16x8 (if 4:2:2) or 16x16 (if 4:2:0)
● 2.b Transform to frequency domain with DCT applied to each macroblock
● 2.c Quantize frequency components. One matrix for luma and one for chrominance
● 2.d Discard high-frequency coefficients due to human perception. This gives the bigger or smaller compression [2nd compression, lossy]
Image files and formats
34 34Multimedia Services:
Image
JPEG Encoding flow (3/3):Final entropy encoding
● 3. Compress the resulting 8 x8 blocks in zig-zag with entropy encoding [3rd compression, lossless]
The matrix from 2.d is like: Path for compression in 3:
(Logo in FFmpeg library: )
Image files and formats
35 35Multimedia Services:
Image
PNG file format● Lossless compression: filter + Deflate algorithm● 3 Type of images: Palette-based, Grayscale, Direct color RGB[A]● Color depth: bits/channel x no. of channels (Gray, R,G,B,A)● Bits/channel: 1, 2, 4, 8, 16 bits/channel (if alpha 64 bits/pixel), e.g.
PNG8: indexed with 8 bits, PNG24: RGB 8 bits/channel, PNG32: PNG24+Alpha● RGB space color● Chunks:
– Criticals: Header (width, height, color type, bit depth), Palette, Data– Others: ICC profile, gamma, text,...
● Interlacing: 2-dimensional, 7-pass● Does not support animation
Image files and formats
36 36Multimedia Services:
Image
TIFF file format
● Can be seen as a file format, and as a container for lossy and lossless data
● Much more powerful and complicated than PNG or JPEG● Applications:
– Base for lot of raw proprietary formats in cameras– Professional image editing
● Allowed compressions:– Lossless: CCITT Group 3 and Group 4 bi-level, LZW– Lossy: JPEG
● Color spaces of images: RGB, CMYK, YCbCr,...
Image files and formats
37 37Multimedia Services:
Image
Formats for image metadata
● Metadata: date and time, geolocation, data about the image (e.g. orientation),...
● Formats that specify how metadata is inserted into standard formats (e.g. JPEG, TIFF,...):– EXIF: Used in digital cameras and scanners– XMP– IPTC IIM
Image files and formats