Image Quality Metrics
• Image quality metrics • Mutual information (cross-entropy) metric
• Intuitive definition • Rigorous definition using entropy• Example: two-point resolution problem • Example: confocal microscopy
• Square error metric • Receiver Operator Characteristic (ROC)
• Heterodyne detection
MIT 2.717 Image quality metrics p-1
Linear inversion modelobject
channel
gHf
hardware “physical attributes”
(measurement) field
propagation detection
inversion problem: determine f, given the measurement g = H f
noise-to-signal ratio (NSR) = 1power) signal (average
variance) (noise = σ 2
= σ 2
normalizing signal power to 1 MIT 2.717 Image quality metrics p-2
Mutual information (cross-entropy)object
channel
gHf
hardware “physical attributes”
(measurement) field
propagation detection
2
1 ln = ∑
n1 eigenvalues µkC
+ of H2σ2k =1
MIT 2.717 Image quality metrics p-3
The significance of eigenvalues
n
n-1
1 0
2σ
2µ 2µ 2µ 2µ
(aka
is worth) ∑
=
+=
n
k
k
1 2
2
2 1C
σ µ
...
...
rank of measurement
how many dimensions
the measurement 1 ln
n n−1 2 1
eigenvalues of HMIT 2.717 Image quality metrics p-4
Precision of measurement2 k
µ 2σ
n1 1 ln +
2 2 2 t t 1 noise floor
−µσµC < <
2∑k 1=
= =
2
2
+
+
σµt1 ln
this term
2 tµ
2σ −
2 t 1 2 +
−µσ
1 ln +
+
1 ln +
2+... ...
≈precision of (t-2)th measurement ≤1
E.g. 0.5470839348
these digits worthless if σ ≈10-5
MIT 2.717 Image quality metrics p-5
this term≈0
Formal definition of cross-entropy (1)
EntropyEntropy in thermodynamics (discrete systems): • log2[how many are the possible states of the system?]
E.g. two-state system: fair coin, outcome=heads (H) or tails (T) Entropy=log22=1
Unfair coin: seems more reasonable to “weigh” the two statesaccording to their frequencies of occurence (i.e., probabilities)
)Entropy −= ∑ p( logstate p( state)2 states
MIT 2.717 Image quality metrics p-6
Formal definition of cross-entropy (2)
• Fair coin: p(H)=1/2; p(T)=1/2
bit11Entropy = − 2
1 1 1log 2
− 2
log2 = 2 2
• Unfair coin: p(H)=1/4; p(T)=3/4
1Entropy = − 4
1 3log 4
− 4
log2 3= bits81.02 4
Maximum entropyMaximum entropy ⇔⇔⇔⇔⇔⇔⇔⇔ Maximum uncertaintyMaximum uncertainty
MIT 2.717 Image quality metrics p-7
Formal definition of cross-entropy (3)Joint EntropyJoint Entropylog2[how many are the possible states of a combined variable
obtained from the Cartesian product of two variables?]
EntropyJoint ( YX ) −= ∑ ∑ yx p )log yx p ), ( , ( ,2 states states
Xx Yy∈ ∈
,object EntropyJointE.g. ( GF )= ?
hardware channel
“physical attributes”
(measurement) field
propagation detection gHf MIT 2.717 Image quality metrics p-8
Formal definition of cross-entropy (4)Conditional EntropyConditional Entropylog2[how many are the possible states of a combined variable
given the actual state of one of the two variables?]
EntropyCond. ( Y | X ) −= ∑ ∑ yxp ) log xyp )( , ( |2 states states
Xx Yy∈ ∈
object EntropyCond.E.g. ( G | F )= ?
hardware channel
“physical attributes”
(measurement) field
propagation detection gHf MIT 2.717 Image quality metrics p-9
Formal definition of cross-entropy (5)object
hardware channel
“physical attributes”
(measurement) field
propagation detection gHf
adds uncertainty to the measurement wrt the objectNoise adds uncertainty⇔ eliminates informationeliminates information from the measurement wrt object
MIT 2.717 Image quality metrics p-10
Formal definition of cross-entropy (6)uncertainty added due to noise
representation by Seth Lloyd, 2.100 Entropy Cond. ( G F )|
Entropy( )F ( ),C G F Entropy( )G information information contained contained
in the object in the measurement
Entropy Cond. ( F G ) cross-entropy | (aka mutual information)
information eliminated due to noise
MIT 2.717 Image quality metrics p-11
Formal definition of cross-entropy (7)
( )FEntropy ( )GEntropy
( ),
( )| ( )|
( ),C
G F Entropy Joint
G F Entropy Cond. F G Entropy Cond.
G F
MIT 2.717 Image quality metrics p-12
Formal definition of cross-entropy (8)
FF GGinformationinformation
sourcesource(object)(object)
informationinformationreceiverreceiver
(measurement)(measurement)Corruption source (Noise)Corruption source (Noise)
Physical ChannelPhysical Channel(transform)(transform)
, |C( GF ) = Entropy(F ) − EntropyCond. ( GF ) |= Entropy(G ) − EntropyCond. ( FG )
,= Entropy(F ) + Entropy(G ) − EntropyJoint ( GF )
MIT 2.717 Image quality metrics p-13
Entropy & Differential Entropy
• Discrete objects (can take values among a discrete set of states) – definition of entropy
( )log2 x pEntropy = −∑ x p ( )k k k
– unit: 1 bit (=entropy value of a YES/NO question with 50% uncertainty)
• Continuous objects (can take values from among a continuum) – definition of differential entropy
( )ln x pEntropy Diff. = − x p ( )dx∫ Ω( )X
– unit: 1 nat (=diff. entropy value of a significant digit in the representation of a random number, divided by ln10)
MIT 2.717 Image quality metrics p-14
Image Mutual Information (IMI)object
channel
gHf
hardware “physical attributes”
(measurement) field
propagation detection
Assumptions: (a) F has Gaussian statistics (b) white additive Gaussian noise (waGn) i.e. g=Hf+w where W is a Gaussian random vector with diagonal correlation matrix
Then C G F,( )=n1 ∑ +
1 ln
µσ2
2 k µk of seigenvalue : H
=2 1k
MIT 2.717 Image quality metrics p-15
Mutual information & degrees of freedom
n
n-1
1 0
2σ
2 nµ 2
1−nµ 2 2µ 2
1µ...
...
rank of measurement
mutual
∑ =
+=
n
k
k
1 2
2
2 1C
σ µ
2σ
H σ2
MIT 2.717
1 ln
information As noise increases • one rank of is lost whenever
overcomes a new eigenvalue • the remaining ranks lose precision
Image quality metrics p-16
Example: two-point resolutionFinite-NA imaging system, unit magnification
Two point-sources Two point-detectors
~ A gAf A A (object) (measurement)
intensities x measured ~ B gBfB B
Classical view
noiseless
x Ag Bg
intensities intensity
emitted @detectorplane
MIT 2.717 Image quality metrics p-17
Cross-leaking power
A ~ B ~
ss
BAB
BAA
fsfg sffg
+=
+=
( )xs 2sinc=
MIT 2.717 Image quality metrics p-18
IMI for two-point resolution problem
+ =11 µ1 ss ( ) H − = 2det 1H =
s − = 11 µ 2 ss
1 − s1 H− 1 =
2 − 11− ss
2( 1 ) 2( 1 )1 ln
1 ln +
1
2
− 1 +( G F ) s sC + +=, 2σ 2σ2
MIT 2.717 Image quality metrics p-19
IMI vs source separation
( ) 2
1SNR σ
=
s → 0s → 1MIT 2.717 Image quality metrics p-20
IMI for rectangular matrices (1)
H = =
H
underdeterminedunderdetermined overdeterminedoverdetermined(more unknowns than (more measurements
measurements) than unknowns)
eigenvalues cannot be computed, but instead we compute the singular valuessingular values of the
rectangular matrix
MIT 2.717 Image quality metrics p-21
IMI for rectangular matrices (2)
HT
H
= square matrix
ˆ T 1 Trecall pseudo-inverse f = ( H H )− H g
inversion operation associated with rank of Tseigenvalue ( H H )≡ aluessingular v ( ) H
MIT 2.717 Image quality metrics p-22
IMI for rectangular matrices (3)object
channel
gHf
hardware “physical attributes”
(measurement) field
propagation detection
under/over determined n1 +
τ k2σ
singular valuesof H1 ln = ∑
C 2k =1
MIT 2.717 Image quality metrics p-23
Confocal microscope
Small pinhole:
Intensity
object beam
splitter
pivirtual slice
detector
nhole Depth resolution
Light efficiency
Large pinhole:
Depth resolution
Light efficiency
MIT 2.717 Image quality metrics p-24
Depth “resolution” vs. noise
point sources, Object structure: mutually
incoherent
optical axis
sampling distance
Imaging method
correspondence intensity measurements
CFM
object scanning direction
NA=0.2
MIT 2.717 Image quality metrics p-25
Depth “resolution”vs. noise & pinhole size
units: Rayleigh distance MIT 2.717 Image quality metrics p-26
IMI summary
• It quantifies the number of possible states of the object that the imaging system can successfully discern; this includes – the rank of the system, i.e. the number of object dimensions that
the system can map – the precision available at each rank, i.e. how many significant
digits can be reliably measured at each available dimension • An alternative interpretation of IMI is the game of “20 questions:” how
many questions about the object can be answered reliably based on the image information?
• IMI is intricately linked to image exploitation for applications, e.g. medical diagnosis, target detection & identification, etc.
• Unfortunately, it can be computed in closed form only for additive Gaussian statistics of both object and image; other more realistic models are usually intractable
MIT 2.717 Image quality metrics p-27
Other image quality metrics
Mean Square Error (MSQ) between object and image•• Mean Square Error (MSQ) 2( f ) ofresultˆ E f − f k == ∑
k k inversionobject
samples
– e.g. pseudoinverse minimizes MSQ in an overdetermined problem – obvious problem: most of the time, we don’t know what f is! – more when we deal with Wiener filters and regularization
•• Receiver Operator ChaReceiver Operator Charracteacterriisstictic– measures the performance of a cognitive system (human or
computer program) in a detection or estimation task based on the image data
MIT 2.717 Image quality metrics p-28
Receiver Operator Characteristic
Target detection task
Example: medical diagnosis, • H0 (null hypothesis) = no tumor • H1 = tumor
TP = true positive (i.e. correct identification of tumor) FP = false positive (aka false alarm)
MIT 2.717 Image quality metrics p-29