Date post: | 05-Apr-2018 |
Category: |
Documents |
Upload: | priyam-gupta |
View: | 239 times |
Download: | 0 times |
of 14
8/2/2019 Confidence Weighting
1/14
Presenter/Author: Scott McCloskeyHoneywell Labs, Minneapolis, MN, USA
Confidence Weighting for Sensor Fingerprinting
8/2/2019 Confidence Weighting
2/14
HONEYWELL PROPRIETARY
Outline of Talk
1. Motivation for Sensor Fingerprinting
2. Review of Chens Method
3. Independent Testing & Analysis
4. Confidence Weighting to HandlePersistent Edges
5. Experimental Results
6. Future Work
8/2/2019 Confidence Weighting
3/14
HONEYWELL PROPRIETARY
Common Source Camera Identification Problem: Given two videos (or sets of images), can we determine whether
or not they were taken with the same camera?
Scenario: Videos of two IED events are posted to YouTube. If they were
taken with the same camera, we establish a link between the events. Applications:forensic data analysis, social network analysis
Signature Data Advantages/Disadvantages
Image/video header data Quick and easy
Easily spoofedModelLevel Identification
Lens distortions Cameras w/ interchangeable/zoom lenses
CFA interpolation Monochrome images/video
Device-Level IdentificationDead pixels, dark noise Typically corrected in-camera
Photo-response non-uniformity(PRNU) of cameras sensor
Device specific Signature space is large Difficult to correct in-camera
8/2/2019 Confidence Weighting
4/14
HONEYWELL PROPRIETARY
Photo-Response Non-Uniformity (PRNU)
Due to material and manufacturing imperfections, eachpixel on a sensor has a slightly different (non-uniform)response to incoming light. This is most noticeable inimages of uniformly-illuminated flat fields.
Step 1: Signature Extraction Step 2: Signature Comparison
1. Separate each frame into scenecontent and noise components.
2. Average noise component is thesignature.
1. Compute cross-correlation ofinput signatures.
2. Measure sharpness of peak
3. Compare to threshold
Algorithm proposed by: M. Chen, J. Fridrich, and M. Goljan in Source Digital Camcorder Identification Using Sensor Photo-Response Non-Uniformity. Proc. of SPIE, January 2007.
Because the magnitude of this noise is related to environmental conditions(temperature) and because most scenes are not flat fields, the non-uniformity is not corrected in camera.
PRNU-based sensor fingerprints can distinguish between a large numberof devices. If we presume only that we can distinguish three levels ofresponse (normal, high, low), the number of signatures for a 1MP sensoris 3
1000000, which is practically infinite.
8/2/2019 Confidence Weighting
5/14
HONEYWELL PROPRIETARY
Signature Extraction
Unlike other applications, where Computer Vision methods to abstractaway differences between cameras to recognize scene objects (faces,etc.), we now need to abstract away differences between scenes and
recognize camera-specific signatures.
Given an input video, we remove scene contentfrom each frame by applying a de-noising methodand subtracting that result from the original.
The maximum likelihood estimate of the PRNUsignature is:
where Ik is the raw frame, Ik is the de-noised frame, Kis the number of
frames, and Pis the signature.
Input
SceneContent
Noise
^
^
8/2/2019 Confidence Weighting
6/14
HONEYWELL PROPRIETARY
Signature Comparison1. Compute cross-correlation of signatures at different scales.
2. Measure the magnitude of the peak using Peak-to-Secondary Ratio(PSR). This is simply the ratio of the heights of the largest and secondlargest peaks in the cross-correlation.
3. Compare the PSR to a threshold that determines whether the twovideos are said to match.
MismatchMatch
Videos from the same camera will have similar PRNU patterns, and theircross-correlation function will appear similar to a delta function. Mismatchedvideos will have dissimilar PRNU patterns, and the cross-correlation will be arandom pattern.
8/2/2019 Confidence Weighting
7/14HONEYWELL PROPRIETARY
Evaluating Chens Algorithm: Test Videos
Testing presented in the original paper was somewhat limited,with little analysis of the results.
In order to understand the strengths and weaknesses of thealgorithm, we test it against a suite of videos which representa wide range of potential inputs: indoor/outdoor scenes
zooming/moving/stationary camera
flat fields, highly-textured scenes
image stabilization
data from camcorders and digital still cameras with video mode
night mode (feature on camcorders) and daylight mode
When available, video data is acquired without compression. All test videos are 30f.p.s. for 40 seconds (K=1200).
Test uncompressed video, as well as XVID-compressedderivatives.
8/2/2019 Confidence Weighting
8/14HONEYWELL PROPRIETARY
Evaluating Chens Algorithm: Results
Aoutdoor, movingB indoor, flat fieldC indoor, tripodD indoor, movingE indoor, movingF outdoor, stabilizationG indoor, movingH indoor, zoomingI indoor, moving (night mode)Jflat field (night mode)X indoor, moving
Key
True Match
True Non-match
False Match
False Non-match
Test Scenes:
8/2/2019 Confidence Weighting
9/14HONEYWELL PROPRIETARY
Problem 1: Digital Image Stabilization
A common feature on most video cameras, imagestabilization compensates for camera motion that may
disorient or nauseate the viewer. Opticalimage stabilization uses a floating lens element to
smooth out camera motion. Not a problem.
Digitalimage stabilization uses sensors to measure cameramotion. Digitized frames are shifted to compensate.
The PRNU estimate relates to the sensitivity of sensor pixels.A pixel location in the video is assumed to correspond to thesame sensor location in each frame. The shifting of framesviolates this assumption.
We are attempting to characterize the extent to whichstabilization can be handled, in terms of the percentage offrames that are shifted.
8/2/2019 Confidence Weighting
10/14HONEYWELL PROPRIETARY
Problem 2: Persistent Edge Content
De-noising has been long studied in image processing, and the problem iswell known to be ill-posed.
Most de-noising methods misclassify some portion of high-frequencyscene content as noise, particularly near edges.
When estimating the signature, then, the area around edges will beproblematic. If the video features stationary objects, as is the case withtripod-mounted cameras, edges appear in the extracted signature.
Edges in the signature can cause mis-classifications, particularly falsenegatives. False positives may also occur, if these spurious edges appearin similar locations in videos from different cameras.
Interview Video
Extracted Signature
8/2/2019 Confidence Weighting
11/14HONEYWELL PROPRIETARY
Confidence Weighting
Chens method treats each pixel of each frame the same,regardless of its content. This conflicts with the intuition that
flat regions of a scene are more useful for PRNU estimation. In light of the relative difficulty inherent in noise separation
near edges, we should endeavor to avoid the inevitableerrors contributing significantly to the estimated signature.
Based on this reasoning, we propose confidence weighting
for sensor fingerprinting. Specifically, we wish to preventerroneous noise estimates near texture/edges from distortingthe estimated signature. Within frames, we weight againstregions likely to produce erroneous noise estimates.
8/2/2019 Confidence Weighting
12/14HONEYWELL PROPRIETARY
Confidence Weighting for Persistent Edge Content
Interview Video
Extracted Signature
Confidence Map
General Idea: Analyze each frame to predict failures of the de-noising method. Use this togenerate a confidence map that weights the contribution of different scene regions to theestimated fingerprint. Low-confidence regions are not allowed to introduce spurious features tothe fingerprint.
Experiments use the confidence weight
where pis a pixel, Gis a Gaussian filter, and is the gradient operator.
8/2/2019 Confidence Weighting
13/14HONEYWELL PROPRIETARY
Experimental Results
Old Method New Method
Aoutdoor, movingB indoor, flat fieldC indoor, tripodD indoor, movingE indoor, movingF outdoor, stabilizationG indoor, movingH indoor, zoomingI indoor, moving (night mode)
Jflat field (night mode)X indoor, moving
Key
True Match
True Non-match
False Match
False Non-match
Test Scenes:
8/2/2019 Confidence Weighting
14/14HONEYWELL PROPRIETARY
Other Applications of Confidence Weighting
We have shown that confidence weighting can be used toimprove the quality of extracted PRNU signatures by
discriminating between regions within frames. The same framework can be expanded to includediscrimination between frames, based on their differing utilityto signature estimation. We plan to investigate two cues: signal amplification (gain) per frame. Cameras adjust to varying light
by modifying the gain, increasing it when illumination decreases.Frames with higher gain will have relatively higher levels of noise, fromwhich PRNU will be better estimated.
keyframe/interframe characterization. Most video compressionformats are heterogeneous, with certain keyframes preserved at ahigher quality. Noise estimated from such frames are likely to be more
useful for PRNU estimation.
In addition to relative discrimination, confidence measurescan be used to determine when the extracted signature issufficient, or whether more/better frames are needed.