Computational colorLecture 1
Ville Heikkinen
1. Introduction
- Course context- Application examples (UEF research)
2
Standard lecture course:
- 2 lectures per week (see schedule from Weboodi)- exercises
Course page (available in Moodle soon…):
- lecture slides- exercises + data
3
Course
Color in image processing
4
Computations
Measurement devices produce data that are usually
represented with vectors, matrices, and tensors.
Efficient image analysis and processing usually requires
understanding of color/spectral data and suitable
computational methods.
5
Biometry, medical diagnosis, security
6
Is illumination same in all images?
Are images calibrated accurately?
Source of color data
In this course we assume that measurement devices
include
standard color imaging devices (e.g. in mobile phone)
and
spectral imaging devices
(multispectral- and hyperspectral cameras).
7
Color (trichromatic, RGB) imaging devices
8
Example: 3-dimensional vectors
(pixelwise RGB vectors corresponding to a digital color image)
9
Is color enough?Color representation
Surface reflectance (r)
Wavelength [nm]
10
A simple spectral imaging system
11
A complex spectral imaging system
12
Spectral image as a data structure
13
14
Reflectance spectrum corresponding to a pixel
15
Sampling
Images corresponding to spectral bands
16
Applications for spectral data and calibrated color
Biomedical imaging
Biometry and security
Color analysis, display, and printing
Cultural heritage imaging
Environmental monitoring
Industrial machine vision
17
18
Course context: Color management and data analysis...
19
...where the main phases are
1. Measurement
2. Processing
3. Accurate color representation
4. Data analysis (e.g. machine learning)
The main topics in this course are:
1. Vector space formulations for colorimetry. Relation between spectral spaces and color spaces. Fundamental colorimetry
2. Estimation of standard color values from device responses
3. Estimation of spectral data from device responses
4. Representation of spectral data using subspaces
5. Implementation of computational models using MATLAB orsome other computational environment.
20
Course outline
Practical/project work
• Data + small analysis task
• Students write a report of obtained results. The reports are written as a form of (approx. 6 page) ”scientific publication” consisting of
• Abstract
• Introduction
• Methodology
• Experiments
• Discussion
• Conclusions
21
Example 1
Computational spectral imaging:
Increasing the availability of spectral data for common people (as well as for experts)
22
A (real) hyperspectral imaging system (100k), UV-VIS-NIR
“A Wide Spectral Range Reflectance and Luminescence Imaging System”,T. Hirvonen et al., Sensors, Vol. 13(11), 14500-14510, 2013. 23
Properties of hyperspectral imaging
+ h• Narrow spectral bands• VIS, IR and UV• Estimation of reflectance information can be done easily by using
reference surfaces.
-• jCosts• Measurement speed (slow)• Light source properties (heat)• Spatial properties of obtained images• High-level of expertise• Practicality of measurement setting• Several systems are not mobile
24
25“Spectral imaging using consumer-level devices and kernel-basedregression”, V. Heikkinen et al., Journal of the Optical Society of America A,Vol. 33(6), 2016.
Multispectral imaging system (1k) + mathematics + programming
-> Estimated hyperspectral image in visual wavelength range
Properties of multispectral imaging
+ h• Fast imaging (moving objects, video imaging)• Inexpensive (if based on RGB technique)• Large spatial resolution• Low demands for light source• Practical• RGB based systems are in standard use in several applications• Mobile
-• Estimation of reflectance information is not trivial• Possibly broad spectral bands.• Sensitivity of RGB-based system may be restricted to VIS region
26
Spectral images
visualized
as sRGB
Estimates
visualized
as sRGB
“Spectral imaging using consumer-level devices and kernel-basedregression”, V. Heikkinen et al., Journal of the Optical Society of America A,Vol. 33(6), 2016.
27
28
Comparison of images using sRGB color representation
3 line scans (Specim V10) RGB + Laptop system usingDigital colorchecker in training Training data :
“Spectral imaging using consumer-level devices and kernel-basedregression”, V. Heikkinen et al., Journal of the Optical Society of America A,Vol. 33(6), 2016.
29
Comparison of images using PCA eigenimages
3 line scans (Specim V10) RGB + Laptop system usingDigital colorchecker in training
3 line scans (Specim V10) RGB + Laptop system usingDigital colorchecker in training Training data :
Supplements for “Spectral imaging using consumer-level devices and kernel-based regression”, V. Heikkinen et al., Journal of the Optical Society ofAmerica A, Vol. 33(6), 2016.
30
What kind of training data are needed for the imaging system?
3 line scans (Specim V10) RGB + Laptop system usingDigital colorchecker in training
“Spectral imaging using consumer-level devices and kernel-basedregression”, V. Heikkinen et al., Journal of the Optical Society of America A,Vol. 33(6), 2016.
31
What kind of applications there are?
3 line scans (Specim V10) RGB + Laptop system usingDigital colorchecker in training
“Spectral imaging using consumer-level devices and kernel-basedregression”, V. Heikkinen et al., Journal of the Optical Society of America A,Vol. 33(6), 2016.
Example 2:
Spectral imaging as a tool for object analysis and sensor development
32
Hyperspectral imaging as tool for tree seeds screening
33
Line scan imaging in 400-2500 nm using two cameras.
Tree seeds in three classes
”Thermal and hyperspectral imaging for Norway spruce (Picea abies) seedsscreening”, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers andElectronics in Agriculture, Vol. 116, pp. 118-124 , 2015.
Hyperspectral imaging and feature extraction…
34
Line scan imaging in 400-2500 nm using two cameras.
Tree seeds in three classes
”Thermal and hyperspectral imaging for Norway spruce (Picea abies) seedsscreening”, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers andElectronics in Agriculture, Vol. 116, pp. 118-124 , 2015.
…analyzing…feature selection…
35
Line scan imaging in 400-2500 nm using two cameras.
Tree seeds in three classes
”Thermal and hyperspectral imaging for Norway spruce (Picea abies) seedsscreening”, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers andElectronics in Agriculture, Vol. 116, pp. 118-124 , 2015.
...and classifier construction with two indices
(based on three, narrow spectral bands)
36
Line scan imaging in 400-2500 nm using two cameras.
Tree seeds in three classes
”Thermal and hyperspectral imaging for Norway spruce (Picea abies) seedsscreening”, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers andElectronics in Agriculture, Vol. 116, pp. 118-124 , 2015.
37
Example 3. Remote sensing of forest areas
A plot of pine treesin the ground
(10 m x 10 m area)
“Evaluation of simulated bands in airborne optical sensors for tree species identification”, P. Pant et al., Remote Sensing of Environment, Vol. 138, pp 27–37, 2013.
38
Supervised classification of tree species in the ground
“Evaluation of simulated bands in airborne optical sensors for tree species identification”, P. Pant et al., Remote Sensing of Environment, Vol. 138, pp 27–37, 2013.
Example 4. Color calibration in remote sensing
Color characterization for aerial cameras. Susanne Scholz. Applied Geoinformatics for Society and Environment (AGSE) proceedings 2009. 39
Example 5.
Pigment mapping using
spectral reflectance data
(image segmentation)
40
Sensor changes in 1000 nm
Let vector V be a spectral measurement
(121-dimensional vector).
Task:
Find those pixels that have correlation
coefficient > 0.99 with the vector V.
Example: Segmentation of painting image
using correlation coefficient
41
42
43
Task:
Find those pixels that have correlation
coefficient > 0.999 with the vector V.
44
45
46