TECHNICAL DOCUMENTATION

NEED HELP? Call us on +44 (0) 121 231 3215

PAGE 2

TABLE OF CONTENTS

Document Control and Authority.........................................................................................3Introduction.........................................................................................................................4Camera Image Creation Pipeline........................................................................................5 Photo Metadata........................................................................................................6Sensor Identification Technology........................................................................................7 Extraction of Sensor Fingerprint..............................................................................7Forensic Image Identifier.....................................................................................................9 Device Reference Fingerprint.................................................................................10Forensic Image Classifier..................................................................................................11Process Flow for FIA.........................................................................................................12 FIA Project..............................................................................................................12 Default Values for ‘Advanced Options....................................................................13 Types of Images Supported....................................................................................14 Quality of Images....................................................................................................14Operational Computation..................................................................................................16 SPN Fingerprint Extraction....................................................................................16 Identifier Processing...............................................................................................16 Classifier Processing.............................................................................................17Appendix A.........................................................................................................................18AppendixB..........................................................................................................................19Company Overview.............................................................................................................20

DOCUMENT CONTROL AND AUTHORITY

Approved By FPL Board of Directors Authorised Signatory/Company Officer:

________________________________________

Circulation Date: 05 February 2016Version Number: v1.05

PAGE 4

Forensic Pathways has undertaken research to develop a technique for assisting in establishing whether a particular digital image had been produced by a particular camera, camcorder or mobile phone. The outcome of this work is the Forensic Image Analyser (FIA) application, which is a system that performs source imaging device identification. FIA can extract the digital fingerprint of digital photos, using image sensor identification technology, and link them to the camera that took these photos. There are two sections to FIA:

1. Forensic Image Identifier• Creates a camera reference fingerprint• Performs matching of digital photos against the camera reference fingerprint

2. Forensic Image Classifier• Groups a set of photos according to the respective cameras that created these

photos

The imaging devices that are supported by FIA are:• Digital cameras• Mobile (Cell) phones, smart phones with inbuilt cameras• Tablets with inbuilt cameras• Camcorders

The list above is not exhaustive, FIA supports any device that creates digital photos using a camera. FIA supports the two main file formats for digital photographs namely TIFF and JPEG format. If the image is in RAW format, it will need to be converted to either JPEG or TIFF format.

The FIA is a Windows based application which runs on Windows 7, Windows 8.X and Windows 10 in 32/64 bit. The minimum system requirements to run FIA are: • 1 gigahertz (GHz) or faster 32-bit (x86) or 64-bit (x64) processor • 1 gigabyte (GB) RAM (32-bit) or 2 GB RAM (64-bit) • Hard disk space dependent on the amount of images being processed Note that the processing of FIA is parallelised and systems with more processor cores will benefit from the parallelisation and perform faster. The system running FIA needs a pdf file format reader application. The two links below provide free pdf readers: • https://get.adobe.com/uk/reader/ • https://www.foxitsoftware.com/products/pdf-reader/

INTRODUCTION

TECHNICAL DOCUMENTATION // INTRODUCTION

The physical structure of a digital camera is similar to that of a traditional film camera, whereby the light passes through the lens and is captured inside the camera. The main difference is the technology used to transform the light received to an image, using film or digital sensor for a film camera or a digital camera respectively.

Figure 1 shows a simplified version of a digital camera pipeline, with the lens, image sensor and software processing in the camera. The light enters the camera through an aperture in the lens and the light intensity is captured on the sensor. The colours in the light are caught by the CFA (Colour Filter Array) filter, because the sensor stores a grayscale version of the captured image. The camera software processing convert the grayscale image into a digital form and incorporate the image colours back into the digital image. The digital photograph created is then saved either to the camera’s memory or to an external memory card.

The three main file formats for digital photos are RAW, TIFF and JPEG respectively. Mid-range and top-end digital cameras can save the RAW and TIFF digital image to the memory of the camera. All digital cameras, including top-end ranges, have the ability to store JPEG images. The RAW and TIFF file formats allow the digital photograph to store all the information captured by the sensor. Whereas, the JPEG format uses a compression technique to make the file size of the digital photo smaller and thus allows more photos to be stored in the memory storage. The JPEG compression eliminates image information that is not sensitive to the human eye.

At each stage of the camera processing pipeline some artefacts are embedded in the digital image. These artefacts are formed by specific physical properties of hardware components

TECHNICAL DOCUMENTATION // INTRODUCTION

Figure 1: Simplified digital camera pipeline

CAMERA IMAGE CREATION PIPELINE

PAGE 6

and by software properties of the software processing components. The artefacts can be extracted as digital signatures from the photos. FIA uses a sensor identification technology to extract the sensor digital signature (digital fingerprint) from a digital photo.

PHOTO METADATAThe metadata (Exif data) attached to JPEG photos can be used to identify the make and model of the camera that created the photo. An example of an image with its metadata is shown in Figure 2. The metadata may contain the ISO rating, Exposure time and whether the zoom has been fired among other identifying features of the image.

Although the metadata can be used to give an indication of the camera make and model, it is not a reliable method of device identification. The metadata can easily be altered or removed by users or applications. Some online social networking sites (e.g. Facebook) strip the metadata of photos uploaded by users to the site.

FIA does extract the metadata of photos while performing the SPN fingerprint extraction, but the metadata is not used in the process of identification or classification of photos. The metadata is provided as an informative component for the user in the classifier’s report.It has been discovered that silicon sensors used in imaging devices (cameras, mobile phones,

Figure 2: Details of Metadata attached with image.

camcorders) have highly discriminating ‘digital signatures’ or ‘digital fingerprints’. When an image is created the ‘digital fingerprint’ created by the silicon chip is automatically embedded in the image at the time the image is created. This technique is recognised in the scientific field of image forensics as an established and accepted method for determining the source device of an image. The sensor identification technology that is used in FIA is the Sensor Pattern Noise (SPN), which is extracted as the sensor fingerprint of the camera (Figure 3).

Sensor Pattern Noise (SPN) will be used interchangeably with sensor fingerprint throughout the documentation for FIA. SPN occurs due to

the natural imperfections of the silicon material in the light absorbing components in sensors. The sensor fingerprint exists due to the

variations in conversion of light energy to electrical energy in individual pixels of the camera imaging sensor. SPN can differentiate between sensors from the same camera models, e.g. two Canon EOS 1200D cameras will produce different sensor fingerprints. Hence, SPN created by one ‘Chip’ is different to that of other ‘Chips’.

SPN is proportional to the light intensity incident on a pixel, therefore low lighting conditions create weak SPN or no discernible SPN at all. Furthermore, SPN is known not to change over time, i.e. photos taken some years ago with the same camera will produce the same SPN as a photo taken today, provided the same sensor is used in the camera. The SPN fingerprint is spatially distributed over the sensor; thus if the photo is rotated, the SPN pattern will be rotated too and will not match against a non-rotated SPN pattern.

EXTRACTION OF SENSOR FINGERPRINTThe sensor fingerprint extraction from the image is shown in Figure 4. The digital photo is processed in the SPN extractor of FIA, where the SPN is separated from the image content. The sensor fingerprint is saved for the matching process and the image content is discarded.

The SPN fingerprint obtained is often contaminated by strong components in the image, e.g. strong lines and edges. The contamination can prevent the uniqueness of the SPN from becoming the dominant component and can lead to photos from different cameras with similar scene contents having similar fingerprints. Figure 4 shows the contaminated fingerprint with remnants of the stick person in the fingerprint.

TECHNICAL DOCUMENTATION // SENSOR IDENTIFICATION TECHNOLOGY

Figure 3: Sensor fingerprint extracted from

the camera sensor

SENSOR IDENTIFICATION TECHNOLOGY

PAGE 8

Forensic Pathways have developed a unique sensor fingerprint enhancement technique that reduces the strong interfering components from the contaminated fingerprint. The enhancer process cleans the contaminated fingerprint, which results in the unique SPN being the dominant feature of the photograph’s sensor fingerprint.

TECHNICAL DOCUMENTATION // FORENSIC IMAGE IDENTIFIER

Figure 4: Sensor fingerprint separation and extraction from image content. The extracted fingerprint is enhanced to remove contaminations

The Forensic Image Identifier, referred to as Identifier in this document, is a component of FIA where the user has both the evidential photos and an image capture device present. In this scenario, the user can create a camera reference fingerprint (device fingerprint) for the digital camera, which can be used to match against evidential photos.

Figure 5 shows the Identifier process, where a device and a set of images are loaded in FIA. The device is used to create the camera reference fingerprint by using between 20 and 50 uniformly lit, smooth images originating from the device. Examples of smooth images are blue sky photos or blank white wall photos. The Identifier can be used to load an image dataset that will be matched against the device fingerprint and produce a report detailing the results. The identifier can match photos that have been rotated against non-rotated device fingerprints.

The hypothesis that is being tested in the Identifier is:

H0 = Image was not created by the cameraH1 = Image was created by the camera

where H0 is the null hypothesis and H1 is the alternative hypothesis.

TECHNICAL DOCUMENTATION // FORENSIC IMAGE IDENTIFIER

Figure 5: The identifier process showing the device and images being loaded into FIA to create the device fingerprint and photo fingerprints. The Identifier performs the matching and outputs a report

FORENSIC IMAGE IDENTIFIER

PAGE 10

The Identifier finds the correlation match between the camera reference fingerprint and each photo fingerprint in the dataset. If the correlation match value is above a predetermined threshold, the null hypothesis can be rejected. The threshold is pre-set empirically and based on peer reviewed publication in the area of source device identification. If the null hypothesis is rejected, it can be inferred that the photo originates from the camera fingerprint being tested. If the null hypothesis is not rejected, it can be inferred that the photo does not originate from the camera fingerprint under test.

DEVICE REFERENCE FINGERPRINTThe device reference fingerprint of an imaging device (e.g. digital camera, smartphone) can be created by using a set of test photos created by the imaging device. Test photos need to be flatfield smooth with uniformly lit background. A picture of the blue sky or a blank (white or pale coloured) wall. An example of acceptable test photos that can be used to generate the device reference fingerprint is shown in Appendix A.

A selection of between 20 to 50 test images from the device should be loaded in FIA and the required crop size and position are selected. The test images should be cropped to the same size and from the same location as the evidential photos. The device fingerprint is generated by FIA and saved on disk for use in the Identifier. Note that for producing a strong device fingerprint, all of the test images should have the same resolution and the number of test images used should be close to 50.

The Forensic Image Classifier, referred as Classifier in this document, is used in determining whether any two or more images were taken by the same device (camera, mobile phone, or camcorder) but in this case the user only has possession of the images and does not have possession of the imaging device. The technique uses the SPN fingerprint in each of the photos and by comparing them all together, the user can attempt to find evidence to support or negate the hypothesis that the photos grouped together were taken with the same imaging device.

The processing outline of the Classifier is shown in Figure 6. The images dataset can be constructed from the photos extracted from storage devices (laptop drives, memory sticks) and imaging devices or downloaded from the internet (social network sites). These photos are then loaded into FIA and processed through the Classifier. A report is produced which lists the photos grouped according to the imaging device that they originate from.

The output from the classifier, and SPN fingerprint, does not indicate the make or model of the camera that created a photo. FIA has a functionality to extract the metadata (EXIF data) from photos and this metadata information can be displayed as part of the Classifier output report. FIA extracts the metadata from the photos only if the information is present in the photos, some applications and social networking sites (e.g. Facebook) strip the metadata form the photo.

TECHNICAL DOCUMENTATION // FORENSIC IMAGE CLASSIFIER

Figure 6: Image dataset extracted from the devices or downloaded from online, then processed through the Classifier in FIA. The output is a report that groups together photos created by the same imaging devices.

FORENSIC IMAGE CLASSIFIER

PAGE 12

The process flow for FIA, shown in Appendix B, is designed to be as intuitive as possible and allows users to load photos and choose what features they prefer to run. Examples of functions that the users can have in the process flow are:

• Create a new FIA project or load a previously saved FIA project• Load the test images for an imaging device and create a camera reference

fingerprint, if the user has a camera with a set of test images• Load a dataset of digital photos obtained from storage devices or from an

online location

There are two options for the user to process the images; the Forensic Image Identifier and the Forensic Image Classifier. For the Identifier, once the device fingerprint has been created, the matching process can commence with the digital photos dataset. The results are displayed in a PDF report which is saved to disk and can be printed as required.

For the Classifier, there is no device fingerprint present. The user selects the desired images, from the loaded image dataset in FIA, and starts the Classifier. The clusters (groups) of images are displayed in a PDF report which is saved to disk and can be printed as required. The properties of the FIA applications are displayed in the respective output PDF reports.

FIA PROJECTFIA uses the concept of projects to store all references to images and fingerprints as well as details about the properties of the Identifier and Classifier processes. Digital photos and imaging devices that are loaded into FIA are stored in the file structure of the computer. Projects can be created to reference these photos and imaging devices, where the user can have several runs (with different options) and instances of projects for the same images and imaging devices.

A project keeps a reference to the photos repository, both test photos for device fingerprints and evidential photos. The project also stores the properties of the FIA application, for example the image crop size, crop position, similarity matrix size. These properties can be accessed in the ‘Advanced Options’ section within FIA. When a saved project is opened, FIA will adopt the properties in the saved project.

PROCESS FLOW FOR FIA

The project is linked to a single instance of a device fingerprint with a set of properties. If the crop size of the device fingerprint is altered, the previous device fingerprint with the old crop size is overwritten. If the user wants to store both copies of the device fingerprints with different crop sizes, two projects must be created. Each time properties in the ‘Advanced Options’ are changed, they should be saved to a new project.

DEFAULT VALUES FOR ‘ADVANCED OPTIONS’The FIA application starts with the properties set at their default values. The properties are:

• Crop Width and Crop Height • Crop Position • Enhance Image Signature • Voting Pool Size • Similarity Matrix size

The default values are set based on empirical experiments and analysis of peer reviewed publications. The crop width and height are set at 512 pixels each and the crop position is set to the centre of the image. The Enhance Image Signature is set to checked (true), because the SPN fingerprints extracted from images with scene details need to be enhanced. Note that for the device fingerprint creation process, the enhancer is disabled because these images are flatfield images with smooth background. The voting pool size is set to use all the images in the classification process. The similarity matrix for the classifier is set to use all the evidential images loaded, i.e. 100%.

The user has the opportunity to change the default values of the properties in the ‘Advanced Options’ page for FIA. Crop sizes can be set to ‘No Crop’, 128, 256, 512, 1024 pixels. Crop positions can be set to ‘Top Left’, ‘Top Right’, ‘Centre’, ‘Lower Left’, ‘Lower Right’. Enhancer property can be set to checked (enhance) or unchecked (do not enhance). The voting pool size in the classifier can be changed to use the half or one quarter of the images in the training phase of the classification. It is recommended set the voting pool size to ‘All’ for the best classification results. Finally, the similarity matrix size can be changed to use half or one quarter of the total evidential images loaded in the project. Table 1 gives an indication of the recommended similarity matrix size in relation to the number of evidential images being classified, if the user wants to speed up the classifier process.

The different sizes of the similarity matrix for varying amount of evidential images are shown in Table 1. However for increased accuracy of the classification process it is recommended to use ‘All’ (100%) of the evidential images, albeit this will increase the time taken to complete the classification process.

TECHNICAL DOCUMENTATION // PROCESS FLOW FOR FIA

PAGE 14

TYPES OF IMAGES SUPPORTEDThe following photo formats are supported by FIA:

• JPEG• TIFF

RAW image format is supported, provided the image has been converted to TIFF or JPEG format. The TIFF format is a lossless image format that keeps all the details of the components of the RAW image. Photos in TIFF format have larger file sizes than JPEG photos. RAW and TIFF format photos are used mainly by professional and amateur photographers.

Low-end, mid-range and most high-end digital cameras as well as mobile phones and tablets output their photos in JPEG format, which is the de-facto format for compressed digital photos. The JPEG format is a lossy image format that is used to compress photos to produce photos with smaller file sizes. The lossiness associated with the JPEG format means that when images are converted from RAW format to JPEG format, image details that are outside the range of human visual system are suppressed. This produces a smaller image which, to the human eye, looks similar to the original image. The amount of JPEG compression is called compression ratio.

The quality of JPEG images starts to deteriorate if the compression process is too severe. The compression algorithm starts to suppress details within the visual range, horizontal and vertical lines can be seen in the image which is also called blocking artefacts.

QUALITY OF IMAGESThe image quality is an important factor to consider during the SPN fingerprint extraction. The amount of JPEG compression, saturated pixels and dark pixels are features that reduce the strength of SPN in the image and inhibit the SPN extraction. Highly compressed JPEG images weaken the SPN and prevent a reliable extraction of the SPN fingerprint.

SPN which consist mainly of PRNU (Photo Response Non-Uniformity), is proportional to the light intensity incident on the sensor. Therefore, the SPN is stronger in images where brighter

Number of Evidential Photos Percentage Size of Similarity Matrix (%)

150 or less 100

Between 151 to 500 50

More than 500 25

Table 1: Percentage size of similarity matrix in relation to the number of photos being classified.

lights have been absorbed by the sensor. Images with a high percentage of dark (low light levels) pixels will contain a weak SPN.

A similar issue occurs with saturated pixels, where the pixels reach their maximum allowed value. The SPN value cannot be extracted from these pixels. If an image contains a high percentage of saturated pixels, a reliable SPN fingerprint will not be extracted from that image.

Highly compressed JPEG photos or photos that contain large areas of saturated pixels or dark pixels will not provide reliable SPN fingerprints.

TECHNICAL DOCUMENTATION // PROCESS FLOW FOR FIA

PAGE 16

OPERATIONAL COMPUTATION

Experiments were performed on 1000 images of varying resolutions to demonstrate the processing speed of FIA. All the experiments were performed on a Hewlett-Packard laptop running Windows 8.1 with a quad core i5 2.60 GHz processor and 8 GB RAM. The processing of FIA has been parallelised in order to use all the cores on the local machine.

SPN FINGERPRINT EXTRACTIONThe SPN fingerprints were extracted from the 1000 images using FIA. All the images were cropped to the same size from the centre of the image and the time taken to extract the signatures are listed in Table 2.

IDENTIFIER PROCESSINGThe matching of evidential photos in the Identifier is performed in a sequence, which is shown in the process flow diagram in Appendix A. The device reference fingerprint is generated using the test images loaded by the user. The time taken to create the device fingerprints are listed in Table 3 for three different crop sizes using 50 test images.

Table 3: Time taken in seconds to generate device reference fingerprint for 3 different crop sizes using 50 test images from the device.

Table 2: Time taken in minutes to extract the SPN fingerprint from 1000 photos with different crop size in pixels.

The evidential photos can be matched against the device fingerprint generated. An experiment was performed where 1000 photos from 10 cameras, with 100 photos from each camera, were matched against a device fingerprint from one of the cameras in the dataset. The experiment was repeated for three crop sizes. The time taken to perform the matching is shown in Table 4.

CLASSIFIER PROCESSINGIn order to process the SPN fingerprints in the classifier, a similarity matrix needs to be generated based on the correlation matchings between the extracted SPNs. The size of the similarity matrix depends on the number of photos in the dataset. In the experiments that were conducted, the SPNs were randomly chosen from the dataset of 1000 SPNs. The size of the matrices were 300, 500 and 1000 SPNs respectively. The time taken to generate the similarity matrices and complete the classification of the SPN fingerprints are listed in Table 5.

Table 5: Time taken in minutes to classify 1000 photos based on using similarity matrices of difference sizes (300, 500 and 1000), for different crop sizes of the images.

TECHNICAL DOCUMENTATION // OPERATIONAL COMPUTATION

Table 4: Time taken in minutes to match 1000 photos against one device reference fingerprint for three different crop sizes.

PAGE 18

APPENDIX A – EXAMPLES OF IMAGES FOR DEVICE FINGERPRINT

APPENDIX B – FIA PROCESS FLOW

Figure B.1: FIA process flow showing the stages from the start when loading/creating a project to getting the output report for the Identifier or the Classifier.

PAGE 20

Forensic Pathways is an award winning company offering innovative forensic technology solutions to law enforcement and security markets globally. Since its incorporation in 2001 the Company has developed a range of forensic technologies focused on forensic digital forensics, image analysis and the management of cell phone data.

Forensic Pathways’ core technologies include, Forensic Image Analyser (FIA), Volume Image, Analyser (VIA), Forensic Phone Analyser (FPA), and Forensic Digital Exchange (FDX). The development of such software solutions has resulted in the Company winning a number of awards including the Digital Forensic Award 2015, the Digital Forensic Award 2013 and the Orange National Business Award 2009.

The Company is a signatory to the United Nations Global Compact and its CSR Report can be found on the UN Global Compact website https://www.unglobalcompact.org/ along with its own Company website www.forensic-pathways.com

For more information please contact sales@forensic-pathways.com or call +44 (0) 121 231 3215.

Patents:  Patent P331316GBDIv1, GB2467767, GB2486987, EU 2396749, USA8565529 (Canada 275226322 - pending)

COMPANY OVERVIEW

to perform the matching is shown in Table 4. CLASSIFIER PROCESSINGIn order to process the SPN fingerprints in the classifier, a similarity matrix needs to be generated based on the correlation matchings between the SPNs extracted. The size of the similarity matrix depends on the number of photos in the dataset. In the experiments that were conducted, the SPNs were randomly chosen from the dataset of 1000 SPNs. The size of the matrices were 300, 500 and 100 SPNs respectively. The time taken to generate the similarity matrices and complete the classification of the SPN fingerprints are listed in Table 5.

NEED HELP? Call us on +44 (0) 121 231 3215