ENEE631 Digital Image Processing (Spring'09)
Image ForensicsImage Forensics
Spring ’09 Instructor: Min Wu
Electrical and Computer Engineering Department,
University of Maryland, College Park
bb.eng.umd.edu (select ENEE631 S’09) [email protected]
ENEE631 Spring’09ENEE631 Spring’09Lecture 22 (4/22/2009)Lecture 22 (4/22/2009)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [2]
Overview and LogisticsOverview and Logistics
Last Time:
– Fragile data embedding for image forgery detection– Improving watermark robustness via quantization– Robust watermark via spread spectrum embedding
Today:
– Continue on spread spectrum watermark– Digital forensic fingerprinting for traitor tracing– Non-intrusive image forensics
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [4]
Background: Robust Watermark / Data-EmbeddingBackground: Robust Watermark / Data-Embedding
Embedding domain tailored to media characteristics & application requirement
10011010 …
© Copyright …
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [5]
Spread Spectrum Watermark: Spread Spectrum Watermark: Cox et al (NECI)Cox et al (NECI)
What to use as watermark? Where to put it?– Place wmk in perceptually significant spectrum (for robustness)
Modify by a small amount below Just-noticeable-difference (JND)
– Use long random noise-like vector as watermark for robustness/security against jamming+removal & imperceptibility
Embedding v’i = vi + vi wi = vi (1+ wi)
– Perform DCT on entire image and embed wmk in DCT coeff.– Choose N=1000 largest AC coeff. and scale {vi} by a random factor
2D DCT sort v’=v (1+ w) IDCT & normalize
Original image
N largest coeff.
other coeff.
marked image
random vector generator
wmk
seed
1.0nceunit varia zeromean, iid,~iw
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [7]
Cox et al’s Scheme (cont’d): DetectionCox et al’s Scheme (cont’d): Detection Subtract original image from the test one before feeding to
detector (“non-blind detection”)
Correlation-based detection a correlator normalized by |Y| in Cox et al. paper
DCT
compute similarity
thresholdtest image
decision
wmk
DCT select N largest
original unmarked image
select N largest
preprocess
–
YY
WYWYsim
,
,),(
k watermar
watermarkno
:1
:0
NWYH
NYHXXY
–orig X
test X’
X’=X+W+N ?
X’=X+N ?
To think
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [8]
Performance of Cox et al’s SchemePerformance of Cox et al’s Scheme
Robustness
– (claimed) scaling, JPEG, dithering, cropping, “printing-xeroxing-scanning”, multiple watermarking
– No big surprise with high robustness equiv. to sending just 1-bit {0,1} with O(103) samples
Comment– Must store orig. unmarked image “private wmk”/“non-blind” detection– Perform image registration if necessary– Adjustable parameters: N and
Distortion none scale25%
JPG10%
JPG 5% dither crop25%
print-xerox-scan
similarity 32.0 13.4 22.8 13.9 10.5 14.6 7.0 threshold = 6.0 (determined by setting false alarm probability)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [9]
Comments on Cox et al’s SchemesComments on Cox et al’s Schemes
“1000 largest coeff.” before and after embedding– May not be identical (and order may also changes)– Solutions: use orig. as ref; “embeddable” mask to maintain synch.
Detection without using original/host image– Treat host image as part of the noise/interference ~ Blind detection
need long wmk signal to combat severe host interference [Zeng-Liu]
– Can do better than blind detection, as embedder knows the host signal => “Embedding with Side Info.”
./(:blind-non
;/:blind
22
22
www)y
wwy
TdN
dT
N
T
T
H0: <(x + noise), w >H1: < y + noise, w > = < w + (x + noise), w >
vs.H0: < (x + noise) - x, w > = < noise, w > H1: < y + noise - x, w > = < w + noise, w >
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [10]
Improve Invisibility and Robustness on Cox schemeImprove Invisibility and Robustness on Cox scheme
Apply better Human Perceptual Model– Global scaling factor is not suitable for all coefficients
– More explicitly compute just-noticeable-difference (JND) JND ~ max amount each coefficient can be modified invisibly Employ human visual model: freq. sensitivity, masking, …
– Use more localized transform => fine tune wmk for each region
block-based DCT; wavelet transform
Improve robustness: detection performance depends on ||s|| / d Add a watermark as strong as JND allows Embed in as many “embeddable” coeff. => improve robustness
Block-DCT schemes: Podichuk-Zeng; Swanson-Zhu-Tewfik ’97
– Leverage existing visual model for block DCT from JPEG
iiii wJNDvv '
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [11]
Perceptual Comparison: Cox vs. PodilchukPerceptual Comparison: Cox vs. Podilchuk
Original Cox Podilchukwhole image DCT block-DCTEmbed in 1000 largest coeff. Embed to all “embeddables”
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [12]
Compare Cox & Podilchuk Schemes (cont’d)Compare Cox & Podilchuk Schemes (cont’d)
Cox Podilchuk
Amplified pixel-wise difference between marked and original (gray~0)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [13]
Comments on Cox & Podilchuk’s SS WmkComments on Cox & Podilchuk’s SS Wmk
Robustness
– Very robust against additive noise (seen from detection theory)
– Sensitive to synchronization errors, esp. under blind detection jitters (line dropping/addition) geometric distortion (rotation, scale, translation)
Question: How to improving synchronization resilience?
=> add registration pattern; embed in RST-invariant domain; …
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [14]
Localized Embedding: Double-Edge SwordLocalized Embedding: Double-Edge Sword
“Innocent Tools” exploited by attackers: block concealment
Recovery of lost blocks
– for resilient multimedia transmission of JPEG/MPEG– good quality by edge-directed interpolation: Jung et al; Zeng-Liu
Remove robust watermark by block replacement
edge estimation
edge-directed interpolation
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [15]
Block Replacement AttackBlock Replacement Attack
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [16]
Attack effective on block-DCT based spread-spectrum watermark
marked original (no distortion)JPEG 10% after proposed attack
JPEG 10% w/o distort Interp.
w/ orig 34.96 138.51 6.30
w/o orig 12.40 19.32 4.52
512x512 lenna Threshold: 3 ~ 6
Recall: claimed high robustness&quality by fine tuning wmk strength for each region
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [19]
Summary: Spread Spectrum EmbeddingSummary: Spread Spectrum Embedding
Main ideas– Place wmk in perceptually significant spectrum (for robustness)
Modify by a small amount below Just-noticeable-difference (JND)
– Use long random vector of low power as watermark to avoid artifacts (for imperceptibility, robustness, and security)
Cox’s approach
– Perform DCT on entire image & embed wmk in large DCT AC coeff.– Embedding: v’i = vi + vi wi = vi (1+ wi)
– Detection: subtract original and perform correlation w/ wmk
Podilchuk’s improvement– Embed in many “embeddable” coeff. in block-DCT domain– Adjust watermark strength by explicitly computing JND
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [21]
Summary: Type-I Additive EmbeddingSummary: Type-I Additive Embedding
Add secondary signal in host media
Representative: spread spectrum embedding
– Add a noise-like signal and detection via correlation– Good tradeoff between security, imperceptibility & robustness– Limited capacity: host signal often appears as major interferer
modulationmodulation
data to be hidden
Xoriginal source
X’ = X + marked copy
10110100 ...10110100 ...
< X’ + noise, > = < + (X + noise), >
< X’ + noise - X, > = < + noise, >
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [22]
Type-II Relationship Enforcement EmbeddingType-II Relationship Enforcement Embedding
Deterministically enforcing relationship – Secondary information carried solely in watermarked signal– Typical relationship: parity/modulo in quantized features
Representative: odd-even (quantized) embedding– Alternative view: switching between two quantizers w/ step size 2Q
“Quantization Index Modulation”
– Robustness achieved by quantization or tolerance zone– High capacity but limited robustness
even “0”odd “1”
feature value 2kQ (2k+1)Q (2k+2)Q (2k+3)Q
odd-even mapping
lookup table mapping
0 1 0 1
… 0 1 1 0 …
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [23]
Robustness vs. PayloadRobustness vs. Payload Blind/non-coherent detection ~ original copy unavailable Robustness and payload tradeoff Advanced embedding: quantization w/ distortion-compensation
– Combining the two types with techniques suggested by info. theory
RobustnessRobustness
PayloadPayload
ImperceptibilityImperceptibility
stronger noisenoise weaker
-15 -10 -5 0 5 10 15 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
10log10
(E2/2) (dB)
Capacity C
(bits/c
h.
use)
Capacity of Type-I (host=10E) and Type-II AWGN ch. (wmk MSE E2)
Type-I (C-i C-o, blind detection)Type-II (D-i D-o)
-4 -3 -2 -1 0 1
0
0.02
0.04
0.06
0.08
0.1
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [24]
Distortion Compensated Quantization EmbeddingDistortion Compensated Quantization Embedding
Distortion compensation technique– Increase quantization step by a factor for higher robustness– Compensate the extra distortion by dragging the enforced feature
toward the original feature value
Overall embedding distortion unchanged
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Ta
lks
’03
, u
pd
ate
d’0
7-’0
9)
odd/even mapping 0 1 0
X0
Choose alpha to maximize a distortion-compensation SNR
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [25]
Improved Robustness by Distortion CompensationImproved Robustness by Distortion Compensation
– ICS (ideal Costa’s scheme)– SS (spread spectrum additive
embedding)– binary DM (odd-even quantized
embedding)– binary SCS (odd-even quantized
embedding with distortion compensation)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [26]
Issues to AddressIssues to Address
Tradeoff among conflicting requirements– Imperceptibility– Robustness & security– Capacity
Key elements of data hiding– Perceptual model– Embedding one bit– Multiple bits– Uneven embedding capacity– Robustness and security– What data to embed
Up
per
L
ayer
s
Uneven capacity equalization
Error correction
Security
……
Low
er
Lay
ers
Imperceptible embeddingof one bit
Multiple-bit embedding
Coding of embedded data
Robustness
Capacity
Imperceptibility
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [28]
Embedded Fingerprint for Tracing TraitorsEmbedded Fingerprint for Tracing Traitors
Insert special signals to identify recipients
– Deter leak of proprietary documents– Complementary protection to encryption– Consider imperceptibility, robustness, traceability
– Attacks mounted by single and multiple users
Collusion Attack Collusion Attack (to remove fingerprints)(to remove fingerprints)
AliceAlice
BobBob
Colluded CopyColluded Copy
Unauthorized Unauthorized rere--distributiondistribution
Fingerprinted docfor different users
Collusion Attack Collusion Attack (to remove fingerprints)(to remove fingerprints)
AliceAlice
BobBob
Colluded CopyColluded Copy
Unauthorized Unauthorized rere--distributiondistribution
Fingerprinted docfor different users
Extract Extract FingerprintsFingerprints
Suspicious Suspicious CopyCopy
101110 …101110 …
Codebook
Alice, Bob, …
Identify Identify TraitorsTraitors
Extract Extract FingerprintsFingerprints
Suspicious Suspicious CopyCopy
101110 …101110 …
Codebook
Alice, Bob, …
Identify Identify TraitorsTraitors
Multi-user Attacks
Traitor Tracing
President
Satellite Image
Alice
Bob
Carl
w1
w2
w3
LeakLeak
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [30]
Collusion Attacks by Multiple UsersCollusion Attacks by Multiple Users Collusion: A cost-effective attack against multimedia fingerprints
– Users with same content but different fingerprints come together to produce a new copy with diminished or attenuated fingerprints
– Fairness: Each colluder contributes equal share through averaging, interleaving, and nonlinear combining
Collusion byaveraging 1/3
Alice ChrisBob
Colluded copy
Originally fingerprinte
dcopies
Alice Chris
Collage
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [31]
Key IssuesKey Issues
How to construct fingerprints?
– Identify individual users– Resist multi-user collusion
How to embed fingerprints in media data?
– Tailor to media characteristics for robustness & imperceptibility
Interaction between choices of fingerprint construction, embedding, and detection
– esp. to combat collusion attacks– Analogous to “cross-layer” methods in communications
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [32]
Road Map on Media Fingerprinting ResearchRoad Map on Media Fingerprinting Research
Robust EmbeddingOrthogonal Fingerprints
Represent how many users?Resist how many colluders?
“most effective” collusions?
Amount of resources used?
Group-based FP to exploit Attacker Behavior
Coded FP
Joint Coding-Embedding Framework
overcome prior work’s problems of long code length, low resilience, and limited scalability
adapt to media characteristics
Combinatorial codes + CDM
Error correcting codes + TDM
Correlated Fingerprints
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [33]
Example: Orthogonal Fingerprint for Curves/GraphicsExample: Orthogonal Fingerprint for Curves/Graphics Use (approx.) orthogonal sequences as FPs for different users
– Detection by looking for high correlation result
Embed in parametric modeling domain of curve
– Perturb B-spline parameters according to spread spectrum sequences
Detection Statistics
Typical threshold is 3~6 for false alarm of 10-3 ~ 10-9
Original Curve(captured by TabletPC)
Fingerprinted Curve(100 control points)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [34]
Fingerprinting Topographic MapFingerprinting Topographic Map
– Traditional protection: intentionally alter geospatial content
– Embed much less intrusive digital fingerprint for a modern protection
• 9 long curves are marked; 1331 control points used to carry the fingerprint
1100x1100 Original Map Fingerprinted Map
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [35]
Collusion-Resistant Fingerprinting of MapsCollusion-Resistant Fingerprinting of Maps
2-User Interleaving Attack5-User Averaging Attack
. . .
Can survive combined attacks of collusion + print + scan
Can extend to 3-D Digital Elevation Map
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [37]
16-bit Codes for Detecting 16-bit Codes for Detecting 3 Colluders Out of 20 3 Colluders Out of 20
User-1 ( -1,-1, -1, -1, 1, 1, 1, 1, …, 1 ) ( -1, 1, 1, 1, 1, 1, …, -1, 1, 1, 1 ) User-4
Extracted fingerprint code ( -1, 0, 0, 0, 1, …, 0, 0, 0, 1, 1, 1 )
Collude by AveragingUniquely Identify User 1 & 4
Embed fingerprint via HVS-based spread spectrum embedding in block-DCT domain
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [38]
Many Forensic Questions …Many Forensic Questions …
arise from military, intelligence, law enforcement, and commercial applications
What type of sensor was used?
Which camera brand took this picture? What model?
What processing has been done?
– Has it been tampered? manipulated?
What technologies were employed?
– Given two images, are they acquired by devices with similar imaging technologies?
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [39]
Break down the info. processing chain into individual components
Identify algorithms and parameters employed in major components of a digital device or processing system
Exploit Intrinsic Fingerprints via Component ForensicsExploit Intrinsic Fingerprints via Component Forensics
Ref: Swaminathan/Wu/Liu in ICASSP’06 and IEEE Trans. Info Forensics & Security (’07)
Color Filter Array (CFA)
ColorInterpolation
White Balancing
Real world scene Digital imageCamera Components
… Sensors …
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [43]
Forensic Estimation and IdentificationForensic Estimation and Identification
Establish a processing model and estimate parameters– Small # possibilities => exhaustive search or by classifier design – More continuous valued parameters => analyze based on estimation
theory
Example: color interpolation in digital camera– Approximate by texture classification and linear filter
(one set of interpolation coeff. for smooth, horizontal & vertical)
– Find best linear estimate of filter coeff. in each class(least-square type of method for robustness)
– Find CFA pattern in a search space that minimizes fitting errors
CFA Interpolation
R ?
? ?
R ?
? ?
R ?
? ?
R ?
? ?
? ?? ?
CandidateCFA pattern
A x = b Interp. equation set
A ~ non-interp pixels;b ~ interpolated pixels
Interp. coeff. x and fitting error for each region type and color
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [44]
Experiments with Images from Digital CamerasExperiments with Images from Digital Cameras
Camera Model Camera Model
123456789
10
Canon Powershot A75Canon Powershot S400Canon Powershot S410Canon Powershot S1 IS Canon Powershot G6Canon EOS Digital REBELNikon E4300 Nikon E5400Sony Cybershot DSC P7Sony Cybershot DSC P72
111213141516171819
Olympus C3100Z/C3020ZOlympus C765UZMinolta DiMage S304Minolta DiMage F100Casio QV-UX2000FujiFilm Finepix S3000FujiFilm Finepix A500Kodak CX6330Epson PhotoPC 650
19 cameras and 200 image blocks per camera model
– 512 x 512 regions with maximum gradients chosen for analysis (s.t. have substantial revealing evidence on color interpolation)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [45]
Detecting Which Camera Detecting Which Camera BrandBrand Took the Image Took the Image
Canon Nikon Sony Olympus Minolta Casio Fuji Kodak Epson
Canon 96% * * * * * * * *
Nikon * 83% 5% * * * * * *
Sony * * 90% * * * * * *
Olympus * * * 93% * * * * *
Minolta 8% * * * 81% * * * *
Casio * * * 6% * 89% * * *
Fuji * * * * 7% * 87% * *
Kodak * * * * * * * 89% *
Epson * * * * * * * * 100%
Interpolation coefficients as features for classifier Average accuracy: 90% for 9 camera brands on uncontrolled scenes Best related work under controlled, uncompressed setting on input scenes
84% for 3 brands, uncompressed [Kharrazi et al’ 05]; 96% for 3 brands [Bayram et al’ 06]
(* denotes values below 4%)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [47]
Tampering DetectionTampering Detection
Explore intrinsic fingerprints left by various processing modules
– To infer the algorithms and parameters employed in various components of the digital device and processing systems
– New traces or vanished old traces suggests potential post-camera operations
Estimated coeff.
From direct camera output
After post-camera filtering
ColorInterpolation
ColorSensors
Scene Optical Lens System CAMERA
Other SoftwareProcessing
ATampering
/ Stego
B
Black: Sony P72; White: Canon Powershot S410
Grey: Classified as other cameras with low confidence
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [48]
Overview of Scanner ModelOverview of Scanner Model
Tri-linear CCD sensors
Tri-linear CCD sensorsHardcopy
graphic data
Lamp & MirrorsLamp & Mirrors
Lens
Tri-linearcolor filter array
Scanner head
Motion systemMotion system
Shift RegisterAmplifier
A/D Converter
Shift RegisterAmplifier
A/D Converter
Post-processing
Interpolation, Color transformation White balancing, Exposure controlNoise reduction,…
Digitalimage
Software operation
Scanning noise
StatisticalStatisticalnoise featuresnoise features
Scanner modelIdentification
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [50]
E.g.: Noise Features from Wavelet AnalysisE.g.: Noise Features from Wavelet Analysis
ScannedImage I
One stagewavelet
decomposition
HHHLLH
Subband STD &goodness of
Gaussian fitting
Statisticalfeatures
f (3)(I), f (4)(I)
Digital photograph Scanner model 1 Scanner model 2
Histogram Mean and STD Gaussian distribution Goodness of Gaussian fitting
Fitting errorFitting error
HH,HL,LH sub-bandsRGB components2x3x3 = 18 features
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [52]
Acquisition Forensics w/ Noise + Interp. FeaturesAcquisition Forensics w/ Noise + Interp. Features
Q1) What type of device was used to capture the image?
94% accuracy in identifying device type
Q2) What brand/model of the device captured the image?
Cellphone cameras: 98% accuracy over 5 brands
Standalone cameras: 90% over 19 camera models from 9 camera brands
Scanners: 93% accuracy over 9 scanner brands
Cell Phone Camera
Standalone Camera
Scanner
Computer Generated
Input Image
Brand/Model
IdentificationAcquisition Device
Type Identification
Further Forensic Analysis
Sony
Samsung
Nokia
AudiovoxMotorola
CanonFujiFilm
CasioMinolta
Epson
Microtek
AcerScan
Canon
Step 1Step 2
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [53]
Suggested ReadingsSuggested Readings
1. I. Cox, J. Kilian, T. Leighton, T. Shamoon: “Secure Spread Spectrum Watermarking for Multimedia'', IEEE Trans. on Image Proc., vol.6, no.12, pp.1673-1687, 1997.
2. M. M. Yeung, F. Mintzer: “An Invisible Watermarking Technique for Image Verification", Proc. of the IEEE Int’l Conf. on Image Processing (ICIP), Oct. 1997.
3. M. Wu and B. Liu: "Data Hiding in Image and Video: Part-I -- Fundamental Issues and Solutions", IEEE Trans. on Image Proc., vol.12, no.6, pp.685-695, June 2003.
4. M. Wu, W. Trappe, Z.J. Wang, and K.J.R. Liu: “Collusion-resistant fingerprinting for Multimedia,” IEEE Signal Proc Magazine, March 2004.
5. Special Issue on Digital Forensics, IEEE Signal Processing Magazine, March 2009.=> Several articles on (non-intrusive) image and video forensics
6. M. Wu and B. Liu: Multimedia Data Hiding, Springer-Verlag, 2003.
7. I. Cox, M. Miller, and J. Bloom: Digital Watermarking, Morgan Kauffman, 2002.
And the related references cited by these publications.
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [54]
Data Hiding in Binary ImageData Hiding in Binary Image
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [55]
Binary Image: A Simple yet Important ClassBinary Image: A Simple yet Important Class
– Scanned documents, drawings, signatures
Social Security E-Files From Princeton EE201 lab material
E-PAD (InterLink Electronics)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [56]
Copyright Protection for E-PublishingCopyright Protection for E-Publishing
Change horizontal and vertical spacing to embed data
– Eyes can not easily identify such changes– “Make it difficult and not worthwhile rather than impossible”
for cheap, high-volume content ~ newspaper, magazine, E-books possible to remove watermark, but why not just pay a bulk
– Embedding may be through additive or enforcement methods
from http://www.acm.org/~hlb/publications/dig_wtr/dig_watr.html
• N.F. Maxemchuk, S. Low: “Marking Text Documents”, ICIP, 1997.
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [57]
Authentic Signatures?Authentic Signatures?
Digitized signatures become popular in everyday life– At least a good interim solution to carry a long tradition
to digital world
Forgery and mis-use of signatures
Clinton electronically signed Electronic Signatures Act - Yahoo News 6/30/00 http://
www.whitehouse.gov/media/gif/bil.gif as of 7/00
E-PAD (InterLink Electronics)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [58]
Challenges on Hiding Data in Binary ImagesChallenges on Hiding Data in Binary Images
Only two levels are available
– Black-white flipping– Minor tuning on the color is not available
Little room for “invisible changes”
– What places can be changed and what cannot
Uneven distribution of changeable pixels
Related to authentication
– Extract hidden data without the use of original copy
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [59]
Identify Flippable PixelsIdentify Flippable Pixels
Flippability score
– Take the human perception into account Based on smoothness and connectivity
– 0~1, with 0 indicating the pixels that should not be flipped
flip-score 0.625 0.375 0.25 0.125 0.1 0.05 0.01
# of pixels 250 32 3 86 382 662 71
(a)
(b)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [61]
Uneven Distribution of Flippable PixelsUneven Distribution of Flippable Pixels Most on rugged boundary
Multi-bit embedding via spatial division– Partition the image into non-overlapping blocks
Embedding rate (per block)
– variable: need side info.– constant: require larger blocks
Two advanced mechanisms to equalize the distribution– Random shuffling– Recent generalized approach: Wet paper codesU
MC
P E
NE
E6
31
Slid
es
(cre
ate
d b
y M
.Wu
© b
ase
d o
n R
ese
arc
h T
alk
s ’9
8-’0
4)
image size: 288x48 red block size: 16x16
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [62]
Shuffling and Block-based EmbeddingShuffling and Block-based Embedding
Shuffling to equalize distribution of flippables (54 blocks)
Divide the image into blocks and hide one bit in each block – Manipulating pixels with the highest flippability scores in the block – Odd-even embedding
To embed a “0”: even number of black pixels To embed a “1”: odd number of black pixels
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [63]
Shuffling-based Embedding and ExtractionShuffling-based Embedding and Extraction
Data embedding
Data Extraction
Marked Marked binary imagebinary image
ShufflingShufflingBlock-Block-based based
embeddingembedding
Inverse Inverse shufflingshuffling
Original Original binary imagebinary image
Data to be Data to be embeddedembedded
KeyKey
ComputeComputeflippabilityflippability
ShufflingShuffling
ShufflingShufflingBlock-Block-based based
extractionextraction
Test Test binary imagebinary image KeyKey
Extracted dataExtracted data
Enhance security
Simple
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [64]
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
# of flippable pixels per block (signature img)
port
ion o
f blo
cks (
x 1
00%
)
before shuffsimulation meansimulation stdanalytic meananalytic stdbefore shuffle
std after shuffle
mean after shuffle
Compare Analysis with Simulation for ShufflingCompare Analysis with Simulation for Shuffling
Simulation: 1000 indep. random shuff.
q = 16 x 16
S = 288 x 48
N = S/q = 18 x 3
p = 5.45%
before shuffle
mean after shuffle std after shuffle
analysis simulation analysis simulation
m0/N (0th bin) 20.37% 5.16x10-5 % 0 % 9.78x10-5 0
m1/N (1st bin) 1.85% 7.77x10-4 % 0 % 3.79x10-4 0
m2/N (2nd bin) 5.56% 5.81x10-3 % 5.56x10-3 % 0.0010 0.0010
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [65]
Application: “Signature in Signature”Application: “Signature in Signature”
– Annotating digitized signature with content info. of the signed document
Each block is 320-pixel large, 1bit / blk.
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [66]
Application: Annotating Binary Line DrawingsApplication: Annotating Binary Line Drawings
10 characters (~ 70bits) are embedded
originaloriginal marked w/ marked w/ “01/01/2000”“01/01/2000”
pixel-wise pixel-wise differencedifference
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [67]
Fragile Watermark for Tamper Detection of DocumentFragile Watermark for Tamper Detection of Document
Embed pre-determined pattern or content features beforehand Verify hidden data’s integrity to decide on authenticity
(f)
alter(a)
(b)
(g)
after alteration
(e)
(c)
(d)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
ba
sed
on
Re
sea
rch
Ta
lks
’98
-’04
)
ENEE631 Digital Image Processing (Spring'09) Lec 22 – Image Forensics [68]