Self-Validating Binary Document Images (PhD Thesis)

Self-Validating Binary Document Images

By

Thesis Submitted to The University of Nottingham

for the Degree of Doctor of Philosophy

School of Computer Science

September 2009

Ammar Albakaa

Abstract

Binary printed documents are used in a wide variety of applications, for example

contracts, wills, recommendation letters, etc. Such documents are intrinsically

valuable but as a results may be subject to attacks by forgers who would make

deliberate changes to the documents, for example changing a name or a date in a

contract. However, during transmission (e.g. printing, in the mail, when being

scanned), they will be subject to noise, for example spurious marks or errors in

printing may occur. This thesis presents a methodology for use in the verification of

such documents. The methods used seek to reject deliberately forged documents

while at the same time not rejecting those which have only been altered by the

addition of noise. The thesis demonstrates that current techniques used for other

types of document are either unacceptably expensive or intrinsically unsuited to

binary documents. The methods developed have been tested on a range of

deliberately forged documents as well as on valid documents and are shown to be

robust. Limitations in the methods have been identified and solutions to the

problems are proposed.

Contents

Table of Contents

List of Tables ...................................................................................... 1

List of Figures ..................................................................................... 3

List of Abbreviations ............................................................................ 5

Acknowledgements .............................................................................. 7

Chapter One: Introduction .................................................................... 8

1.1 Overview ...................................................................................... 8

1.2 The Project Objective ..................................................................... 9

1.2.1 The Significance of Alternations .............................................. 10

1.2.2 The Location of Changes ........................................................ 10

1.2.3 The Representation of Preservative Data .................................. 11

1.2.4 The Insertion of Preservative Data .......................................... 11

1.2.5 The Sensitivity of the Verifier ................................................. 12

1.3 Overview of Chapters .................................................................... 12

Chapter Two: Literature Review ........................................................... 14

2.1 Introduction ................................................................................. 14

2.2 Methods to Protect Documents ....................................................... 16

2.3 Data Hiding Techniques ................................................................. 16

2.3.1 Steganography ..................................................................... 17

2.3.2 Digital Watermarking ............................................................ 17

2.3.3 Data Hiding Applications ........................................................ 18

2.4 Digital Watermarks ....................................................................... 20

2.4.1 Watermarking Applications ..................................................... 20

2.4.2 Watermark Visibility .............................................................. 21

2.4.3. Watermarking System .......................................................... 21

2.4.4. Watermarks Robustness ....................................................... 22

2.4.4.1. Fragile Watermarks .......................................................... 23

2.4.4.2. Semi-Fragile Watermarks .................................................. 23

2.4.4.3. Robust Watermarks .......................................................... 24

2.4.5 Imperceptibility vs Robustness ............................................... 24

2.4.6 Attacks on Watermarks ......................................................... 25

2.4.7 Image Watermarks Domains .................................................. 26

Contents

2.4.8 Watermark Properties ............................................................ 27

2.4.9 Watermarks for Document Images .......................................... 27

2.4.10 Comparison between Binary Image Watermarking Techniques ... 29

2.5 Methods in Literature .................................................................... 31

2.5.1 Colour and Greyscale Image Data Hiding Methods ................... 31

2.5.1.1 Spatial Domain Data Hiding Techniques ........................... 31

2.5.1.2 Frequency Domain Data Hiding Techniques ...................... 36

2.5.1.3 Hybrid Data Hiding Techniques ....................................... 43

2.5.1.4 Other Data Hiding Techniques ........................................ 44

2.5.2 Binary and Half-tone Image Data Hiding Methods ................... 46

2.5.3 Other Techniques.................................................................. 57

2.6 Conclusion ................................................................................... 60

Chapter Three: Fundamental Methodologies and Principles ....................... 61

3.1 Introduction ................................................................................. 61

3.2 Print and Scan Model ..................................................................... 61

3.2.1 The Print Process .................................................................. 62

3.2.2 The Scan Process .................................................................. 64

3.3 Centre of Gravity .......................................................................... 65

3.4 The Method of Moments ................................................................. 67

3.5 Quad Tree ................................................................................... 68

3.6 The 2D Data Matrix Barcode ........................................................... 70

3.7 Type I and Type II Statistical Errors ................................................ 72

Chapter Four: Methodology ................................................................. 74

4.1 Introduction ................................................................................. 74

4.2 Method 1 ..................................................................................... 74

4.2.1 The Creation of Self-validating Documents ............................. 74

4.2.2 The Verification of Self-validating Documents ......................... 77

4.3 Method 2 ..................................................................................... 82

4.3.1 The Creation of Self-Validating Documents ............................. 82

4.3.2 The Verification of Self-validating Documents ......................... 88

4.4 The Extended Version of Method 2 .................................................. 94

4.5 A comparison between Method 1 and Method 2 ................................. 97

4.5.1 Similarities of Method 1 and Method 2 ..................................... 97

Contents

4.5.2 Differences of Method 1 and Method 2 ..................................... 98

4.6 Limitations of using Method 1 ......................................................... 98

4.7 Modifications to Method 1 ............................................................ 102

Chapter Five: Experimental Results and Analysis .................................. 103

5.1 Overview ................................................................................... 103

5.2 The Experimental Results of Method 1 ........................................... 103

5.2.1 Character Image Analysis .................................................... 104

5.2.2 Results of the Verification System of Method 1 ........................ 115

5.3 The Experimental Results of Method 2 ........................................... 118

5.4 The Experimental Results of the Extended Version of Method 2 ......... 121

5.5 A Comparison between the Results of the Proposed Methods ............. 123

5.6 The Influence of Rotation on the Verification Process ....................... 124

5.7 Invisible Data Hiding ................................................................... 125

Chapter Six: Conclusions and Future Research ..................................... 128

6.1 Recommendations to Improve the System ...................................... 128

6.1.1 Noise Removal ................................................................. 128

6.1.2 De-skewing ..................................................................... 129

6.1.3 Optical Character Recognition (OCR) ................................... 130

6.1.4 Data Compression ............................................................ 131

6.1.5 Data Encryption ............................................................... 131

6.1.6 Auto Selection of Data Embedding ...................................... 132

6.2 Conclusions ................................................................................ 133

Appendices ...................................................................................... 135

Appendix A: Two Sample Document Images Divided by Method 1 ...... 136

Appendix B: Two Sample Document Images Divided by Method 2 ...... 137

Appendix C: Two Sample Document Images Divided by The Extended Version of Method 2 .......................................................... 138

Appendix D: A Sample RYANAIR Flight Boarding Pass....................... 139

List of References ............................................................................. 140

1

List of Tables Table 2.1: Comparison between Steganography and Digital Watermarking ............ 18

Table 2.2 : Applications of Digital Watermarking ................................................ 21

Table 2.3 : A Comparison between Several Watermarking Techniques for Document

Images ........................................................................................ 30

Table 3.1: Hypothesis Test in Statistics ............................................................ 73

Table 4.1 : Possibilities and Results of the 2-bit Representation Comparison .......... 92

Table 5.1 : The Influence of Print-Scan Operation on the Centroids and the Rates of

Black Pixels of 71 Characters Written in 7 Different Font Styles .......... 107

Table 5.2 : The Average Influence of Print-Scan Operation on Characters Written in 7

Different Font Styles Sorted from the Least to The Most Affected Fonts 108

Table 5.3 : The Maximum and Minimum Change in the X, Y, and R Values and their

Most and Least Influenced Characters ............................................. 109

Table 5.4 : The 10 Most Affected Characters by Print-Scan Operation and the

Average Difference in Their Centroid Positions and the Rate of Black Pixels

................................................................................................ 109

Table 5.5 : The 10 Least Affected Characters by Print-Scan Operation and the

Average Difference in their Centroid Positions and the Rate of Black Pixels

................................................................................................ 110

Table 5.6 : The Frequency Percentage of the 10 Most Affected Characters by Print-

Scan Operation ........................................................................... 111

Table 5.7 : The Influence of Print-Scan Operation on the Centroids and the Rates of

Black Pixels of 71 Characters Written in 9 Different Font Sizes ........... 113

Table 5.8 : The Average Influence of Print-Scan Operation on the Centroids and

Rates of Black Pixels of Characters Written in 9 Different Font Styles .. 114

Table 5.9: The Experimental Results of Method 1 Verificatoin System on 55 Unaltered

and 100 Forged Scanned Documents .............................................. 116

2

Table 5.10: The Statistical Error of the Verification System of Method 1 .............. 118

Table 5.11 : The Experimental Results of the Verification System of Method 2 on

Scanned and Forged Images ......................................................... 119

Table 5.12: The Statistical Error of the Test Results of Methd 2 .......................... 120

Table 5.13: The experimental Results of the Verification System of the Extended

Version of Method 2 on Scanned and Forged Images ........................ 122

Table 5.14: The Statistical Error in the Verification System of the Extended Version of

Method 2 .................................................................................... 123

Table 5.15: The Average Rates of Successfully Verified and Rejected Documents by

the Three Proposed Methods After Print-Scan Operation .................... 124

Table 5.16: The Rotation Influence on Document Verification ............................ 125

Table 5.17 : The Percentages of Frequencies of (i & j) Letters per 1000 Words and

Per Page in a Sample (2,188,153 words) Text Book .......................... 127

3

List of Figures

Figure 2.1 : The Block Diagram of Watermark Embedding Scheme ....................... 44

Figure 2.2 : The Block Diagram of Watermark Detection Scheme ......................... 44

Figure 3.1 : The Print-Scan Model .................................................................... 62

Figure 3.2 : The Structure of Laser Printer ........................................................ 63

Figure 3.3: The Structure of Scanner ............................................................... 65

Figure 3.4: The Centre of Mass of (a) Symmetric Shapes and (b) Asymmetric Shapes

.................................................................................................... 65

Figure 3.5: Closed Polygon with 6 Vertices ........................................................ 66

Figure 3.6: Self Intersecting Polygons .............................................................. 67

Figure 3.7: An Example of Quad Tree of an Image (a) before and (b) after

partitioning ..................................................................................... 69

Figure 3.8 : The Depth Levels of Quad Tree Partitioning of an Image .................... 69

Figure 3.9: The 4 Components of the 2D Data Matrix Barcode ............................. 71

Figure 4.1 : An Example of Computed Centroid Point (Xc, Yc) & the Rate of Black

Pixels of a Binary Image Contains the Letter “O”. ................................. 76

Figure 4.2 : A Block Diagram of Creating a Self-Validating Document ................... 77

Figure 4.3: A Sample Scanned Image (a) before & (b) after Noise Removal Process 78

Figure 4.4 : An Example of a Character (a) before and (b) after Forgery, its Centroid,

and Black Rate ................................................................................ 80

Figure 4.5 : The Verification Process of a Scanned Document .............................. 81

Figure 4.6 : An Example of a (10×10) Image where Tmargin = 10 in (a) & Tmargin

= 20 in (b) ..................................................................................... 83

Figure 4.7 : An Example of the Data Compression Technique Used to Convert the 8-

bits/value into 2-bits/value data stream .............................................. 86

Figure 4.8 : A Flowchart of the Creation Process of Self-validating Documents ....... 87

Figure 4.9: An Example of Selected Area of Interest .......................................... 88

4

Figure 4.10: An Example of Cropping an Image With No Registration Marks Using

Method 2 (A) Before Cropping (B) After Cropping ................................. 89

Figure 4.11: A flowchart of the verification process of self-validating documents .... 93

Figure 4.12 : An Example of Two Different Sub-images and Their Vertical &

Horizontal Distributions .................................................................... 96

Figure 4.13: A Sample of a Strikethrough Text .................................................. 99

Figure 4.14 : A Sample of a Strikethrough Text ................................................. 99

Figure 4.15: Texts Written in (a) Arabic, (b) Urdu, and (c) Farsi languages ......... 100

Figure 4.16: Samples of Texts Written in English Script Fonts............................ 100

Figure 4.17: A Sample of Text Image Written in Italic Style and How it is Divided into

Sub-images Using Method 1 ............................................................ 101

Figure 4.18: A Sample of Text (a) Neighbouring a Vertical Line (b) in a Table With

Solid Lines .................................................................................... 101

Figure 5.1: Samples of Connected Characters Due to the Additional Noise of Print-

Scan Operation ............................................................................. 104

Figure 5.2: Samples of Calibri, Arial, Latha, Verdana, Times New Roman, Implact and

Georgia fonts ................................................................................ 105

Figure 5.3: The Average Influence of Print-Scan Operation on the Centroids and

Rates of Black Pixels of Characters Written in 7 Different Font Styles .... 108

Figure 5.4: The Average Influence of Print-Scan Operation on the Centroids and

Rates of Black Pixels of Characters Written in 9 Different Font Styles .... 114

Figure 5.5: The Rotation Influence on Document Verification ............................. 125

Figure 5.6 : An Example of Dot Shifting Technique to Embed 3 Bits of Data by

Shifting the Dot in the Letter (i) ....................................................... 126

Figure 6.1: A Flowchart for the Auto-Selection of Data Embedding Technique ...... 133

5

List of Abbreviations

2D ................. Two Dimensional

AFID ................. Anti-Forgery Identification Document

ASCII ................. American Standard Code for Information Interchange

AWST ................. Authentication Watermarking by Self Toggling

BCH ................. Bose-Chaudhuri-Hocquenghem

BMP ................. BitMaP

CAD ................. Computer Aided Design

CCD ................. Charge Coupled Device

CCW ................. Counter ClockWise

CW ................. ClockWise

CWDD ................. Curvature-Weighted Distance Difference

D⁄A ................. Digital-to-Analogue

DCT ................. Discrete Cosine Transform

DES ................. Data Encryption Standard

DHC ................. Data Hiding Capacity

DHST ................. Data Hiding by Self Toggling

DPI ................. Dots per Inch

DRDM ................. Distance Reciprocal Distortion Measure

DRM ................. Digital Rights Management

DS ................. Digital Signature

DWT ................. Discrete Wavelet Transform

ECC ................. Error Correction Code

FFT ................. Fast Fourier Transform

GIF ................. Graphics Interchange Format

HMAC ................. Hashed Message Authentication Code

IDCT ................. Inverse Discrete Cosine Transform

IDWT ................. Inverse Discrete Wavelet Transform

LSB ................. Least Significant Bits

MAC ................. Message Authentication Code

6

MD5 ................. Message-Digest algorithm-5

MSB ................. Most Significant Bits

MTF ................. Modulation Transfer Function

OCR ................. Optical Character Recognition

PC ................. Personal Computer

PDF ................. Portable Document Format

PSNR ................. Peak Signal-to-Noise Ratio

Q-tree ................. Quad Tree

RGB ................. Red-Green-Blue

RLE ................. Run Length Encoding

RSA ................. Rivest, Shamir, & Adleman

SARI ................. Self-Authentication-and–Recovery Image Watermarking System

SPIHT ................. Set Partitioning in Hierarchical Trees

SysCoP ................. System for Copyright Protection

TIFF ................. Tagged Image File Format

TNR ................. Times New Roman

UDWT ................. Undecimated Discrete Wavelet Transform

WPC ................. Wet Paper Coding

ZMM ................. Zernike Moments Magnitude

7

Acknowledgements

It is a pleasure to thank those who made this thesis possible. First of all, I would

like to thank my father Tahir Albakaa and my mother Koulod Albadri for their unlimited

support. They were always beside me when I needed financial help and incorporeal

support. Dr. Dave Elliman was my supervisor when I started my PhD who helped me

to choose the research topic. Dr. Peter Blenchfield became my supervisor after Dave’s

retirement. I would like to thank both Dave and Peter for sharing thoughts with me,

for their time, encouragement, guidance, advice, and patience. This thesis would not

have been possible without them. Also, I would like to show my gratitude to my friends

Nawfal A. Mehdy, Belal Al-khateeb, Musamer Shamil, and Sanar Shamil for their help

and support. At last but not least, I would like to acknowledge the staff members of

the School of Computer Science and The International Office at the University of

Nottingham.

Chapter 1: Introduction

8

Chapter One: Introduction

1.1 Overview

Many documents are created or converted and then stored in digital format

such as Portable Document Format (PDF) or Microsoft Word Document (DOC) files or

any other digital format. Digital documents are often converted to hardcopy

documents when required. Printed documents with significant information such as

birth certificates, recommendation letters, prescriptions, contracts, and sale receipts

are subject to forgery.

However, it is not likely to validate hardcopy versions of documents unless they

are already printed as machine readable documents. The technology to create a

machine readable document is expensive and used only to protect documents of high

importance or financial value such as bank notes and passports. The price of using

high cost technology to protect ordinary documents such as a letter of

recommendation would not be recovered (Sun et al, 2001).

The simplest way to check the integrity of a printed document is to compare it

with the original copy. However, the requirement for an original copy of each

document during the verification process is unreasonable. Therefore, it is better to

embed an information digest about the whole document, or parts of the document

which may be the subject of counterfeiting, into the document itself. A data hiding

method would be useful in making the document carry this information digest.

Most documents are printed in black and white rather than colour due to the

wide use and availability of black and white printers in offices and the work place

(Zhu, 2003). Hiding information in binary document images is a very challenging

operation and most of the data hiding techniques available are designed for


9

greyscale and colour images because of the fact that any alternation in the binary

image grid can easily cause perceptual artefacts in the modified image. In addition,

the data hiding capacity in binary images is low compared to that in colour or

greyscale images. Furthermore, data hiding methods are very sensitive to D⁄A

conversion which can easily remove or corrupt the embedded data. (Furht et al,

2005; Yang and Kot, 2004; Lu et al, 2003; Mei et al, 2001; Low et al, 1998; Arnold,

2003; Kim and Oh, 2004).

1.2 The Project Objective

The aim of this work is to provide a mechanism to determine the validity of black

and white printed documents. This mechanism must be cost effective compared to

the value of the documents. The degree to which the validity can be verified will

depend on an intrinsic/extrinsic value of the binary documents. This validation will be

of documents which cannot already be guaranteed by security methods.

To create a verification system which is able to check the integrity of document

images after the print-scan operation, it is essential to take the following points into

consideration in order to detect tampering in document images.

1- The significance of alternations in the document to be considered as a

forgery.

2- The locations of the altered parts.

3- The preservative data extracted from the original document that represent

the protected parts of the document. The preservative data are selected

information in the document which counterfieters are interested to alter for

malicious purposes. Examples of the preservative data are someone’s name

or singnature.


10

4- The information hiding or insertion method to embed the preservative data

into the document.

5- The sensitivity measure of the method against minor changes which may

occur in the document before and during the print-scan operation.

The above-mentioned points will be answered and discussed in detail in this

chapter.

1.2.1 The Significance of Alternations

If a hardcopy document has been altered, it is important to measure how

significant the change is to determine whether this document has been forged or not.

The consequences of the failure of detecting significant modifications could be severe

if the forged document were a financial paper such as a contract or were an official

document such as a university certificate or a recommendation letter. This wrong

approval can cost a substantial loss of money in banks and official organizations. In

statistics, this failure is called a Type II error, and is also known as a false-negative

error (Onwuegbuzie and Daniel, 2003). In the case of document authentication, it

means the endorsement of a fake document as a genuine one. The probability of

failure has to be very low in order to achieve a satisfactory rate of reliability.

Otherwise, the authentication system will be defective.

1.2.2 The Location of Changes

If a scanned document has been modified, it is important to locate the position

of parts that have been significantly altered to distinguish between forged and

genuine areas in the document. It makes it easier for the person who verifies the

scanned document to locate where exactly the changes are.


11

1.2.3 The Representation of Preservative Data

Most documents which need to be protected contain vital information which

counterfeiters have an interest in changing. Examples of text that can be a forger’s

target might be an amount of money, a name of person, or a signature in a contract

or any legal document. It is unlikely that every single word in the document is in the

area of the counterfeiters’ interest. Only the important information needs to be

encoded and verified and failure to verify these data leads to considering the

document as a forgery.

This significant part of the document must be represented in different ways

depending on many elements such as the data type, data size, the data host, and

the level of robustness required for the verification system. Examples of this data

representation can be a character’s ASCII code, a hash function, or any other

computed values representing details about the data such as the centre of gravity or

the black/white colour ratio of objects or characters in the scanned document. To

create a self-validating document, this preservative data must be inserted in the

document itself.

1.2.4 The Insertion of Preservative Data

The preservative data needs to be stored in the same document where this data

was extracted from, in order to produce a self-validating document. The inserted

data must be extractable after a print/scan operation is applied on the host. Various

methods can be used to embed this data into the document. Data hiding techniques

can be used if the capacity of the host image is sufficient to hold the preservative

data. Visible barcodes attached to the document have also been used in some


12

applications to carry some information about the host. In general, the more

preservative information is stored in the document the more reliable the verification

system will be.

1.2.5 The Sensitivity of the Verifier

When a document is printed and scanned, various levels of noise appear on the

document after the operation. The additive noise can be caused during the print or

scan process even if the glass of the flatbed scanner was completely clean. Also,

printed details in hardcopy documents may fade if kept and exposed to light for a

long time. The mistreatment and negligence by the holders of paper documents will

lead to paper documents sometimes being bent or torn.

Despite the fact that those documents are still genuine, the verification system

may consider them as forged copies because of the unintentional added noise. The

system should be able to distinguish between malicious and innocent changes.

Otherwise, authentic papers may be rejected whilst they are supposed to be

accepted and this kind of false verification is called a Type I Error (false-positive).

Therefore, it is necessary to compromise on the rates of noise to come to a decision

which document is sufficiently corrupted to be considered as a forgery and which one

does not to reach an acceptable level of accuracy in verification.

1.3 Overview of Chapters

This thesis contains six chapters. An introduction about the research area, the

problem of validating binary document images after the print-scan operation, and the

main objective of this research were presented earlier in this chapter. Chapter Two

sheds light on work related to image integrity checks as well as on information hiding

techniques for digital images proposed in the literature. Chapter Three defines some


13

general terminologies and procedures used in the proposed validation system in this

research. The methods to validate black and white hardcopy documents, their

limitations, and modifications are proposed in Chapter Four. The experimental results

of the method proposed in Chapter Four are presented and discussed in Chapter

Five. Furthermore, practical measurements of the print-scan operation impact on

characters in the English language are also discussed in Chapter Five. Chapter Six

concludes the proposed methods and their results and provides recommendations

and suggestions to modify the proposed document validation methods for future

research.

Chapter 2: Literature Review

14

Chapter Two: Literature Review

2.1 Introduction

Document authentication has been an interesting topic for researchers because

of the extensive use of documents in business and society in general. Important

documents, for instance currency notes and passports, can be protected by either

physical means such as added metal security threads or by chemical means such as

special inks, subtle colours, and holograms using expensive high resolution printers.

However, it is impracticable to apply those techniques on some other types of

documents such as certificates, deeds, and contracts because of the high cost of the

materials used in these techniques (Sun et al, 2001).

The widespread availability of low-priced scanners, printers, photocopiers, and

other office equipment facilitates creation and distribution of illicit documents.

Furthermore, paper documents can be converted to digital form easily by using any

Optical Character Recognition (OCR) software without the need for retyping texts

(Arnold, 2003). In addition, data forgery has been made effortless by using one of

the widely-available image editors such as Photoshop. Images can be easily faked by

cropping some regions and substituting them for other parts of the image. This

facilitates undesirable results such as changing names in passports, university

certificates, provincial health cards, or driving licences (Fridrich and Goljan, 1999).

Consequently, the need has increased enormously for protecting important digital

document images used in government institutions and organizations. Many

techniques such as fingerprinting, and encryption have been used to protect sensitive

digital deeds (Alattar et al, 2005). Encryption can prevent illegal access to digital

data. However, the probable collusion of some authorized users and supposed-to-be


15

trusted employees who work in these organizations creates a serious issue as

decrypted data might be copied and distributed illegally as long as these employees

have full access to original documents and they know the decryption key (Furht et al,

2005).

This type of collusion can be made more difficult by one of the aforementioned

methods. Fingerprinting is one proposed solution for finding the source of a leak that

distributes sensitive documents and it should be applied alongside the above

mentioned method to create an efficient security system for digital papers. The main

objective of fingerprinting is to identify any employee who circulates a secret

document outside the workplace. This identification is done by inserting information

into the document itself such as an identification number referring to a particular

employee who was given a document (Alattar et al, 2005).

Moreover, digital multimedia such as images, text, audio, and video files have

been widely used and disseminated via the Internet. Therefore, people around the

world have been increasingly relying on using digital data in communication instead

of traditional paper. However, the extensive use of the Internet has facilitated the

distribution of digital data and, as a result, digital data forgery has become a

lucrative industry (Lan and Tawfik, 1999; Yang et al, 2004).

Indeed, intellectual property protection and authentication of digital data are

considered as essential. Digital watermarking techniques have been used for the

purpose of copyright protection. More recently, counterfeiting detection has become

one of watermarking’s important applications (Lan and Tawfik, 1999). Nevertheless,

more research is needed to detect forgery in document images.


16

2.2 Methods to Protect Documents

According to Zhu et al (2003), there are four different main methods to generate

authentic documents:

1- Using either special physical material such as high-resolution printers or

chemical material such as special inks and papers. This is used for special

applications with high protection requirements for instance currency notes

and bank cheques.

2- Fingerprinting: this is used to trace illegal copies and identify the source of

collusion by inserting information about the legitimate user in each copy given

to this client. However, this method requires special high-priced detectors to

extract the hidden labels. Consequently, it can only be used in a limited range

of application for example tickets and cheques.

3- Digital methods: the aim of these techniques is to embed digital signatures,

generated by authorized users using a private key, into paper documents.

Examples include the use of barcodes and watermarks in bills and ID cards.

4- Visual Cryptography and Optical watermarks: Visual cryptography is based on

dividing an image outline into several parts such that it can be constructed

again by merging all the parts. Optical watermarks offer higher visual quality

and embed several layers of data. Both techniques are used for manual

authentication such as in brand protection applications. However, they are not

able to verify printed and scanned documents.

2.3 Data Hiding Techniques

There has been a substantial increase in production and development of data

hiding techniques to protect the copyright and intellectual property of digital

multimedia such as images, music, video, and texts. The set of these technologies is


17

called Digital Rights Management (DRM). In multimedia DRM, the encryption and

watermarking techniques can be used for data authentication, secret

communications, and copy control (Furht et al, 2005).

Both Digital Watermarking and Steganography are data hiding techniques used to

embed digital information into a cover object. However, their characteristics and

application requirements are different.

2.3.1 Steganography

It is a Greek word which means “Secret Writing” (Brown, 1993). The main

objective of steganography is to embed a secret message into the cover object

without causing any visual artefacts to appear in the cover data. Therefore, the

secret message is the most important element in this technique while the cover is a

tool used to hide and protect this hidden message. The security level of the

embedding and extractions as well as the imperceptibility levels must be high in

steganography. However, the level of robustness does not necessarily have to be

high.

2.3.2 Digital Watermarking

By contrast, Digital Watermarking techniques embed digital data in a cover in

order to protect the host data. Unlike steganography, the imperceptibility of the

inserted watermark is not a priority in some watermarking applications. Therefore,

the watermark can be visible or invisible. Watermarks are usually supposed to be

robust and not easy to eliminate (Katzenbeisser, 2000).

Table (2.1) shows the similarities and differences between Steganography and

digital watermarking in terms of visibility, robustness, security of keys, applications,

and the importance of data.


18

Steganography Watermarking

Visibility Strictly Invisible Visible or Invisible

Robustness Fragile Robust or Fragile

Security of keys High level Low/high level

Applications Point to point secret

communications

Copyright protection, Copy control,

Authentication and tracking.

Importance of Data The hidden data is vital The inserted data is used to protect the

host data

Table 2.1: Comparison between Steganography and Digital Watermarking (Katzenbeisser, 2000)

2.3.3 Data Hiding Applications

Most newspapers, books, pictures, music and videos are being converted to

digital form because digital data are easy to publish, faster to transmit, cheaper to

produce, and smaller in size than analogue data or hardcopies. Therefore, digital

data have been increasingly used and published on the Internet. However, anyone

who has access to the Internet can download and may misuse the published digital

materials. Accordingly, it has been essential to find ways to protect digital

multimedia and printed materials, trace illegal copies, and discover the source of

leaks. Data hiding techniques aim to embed a piece of data imperceptibly into

document images to solve many issues. Chen et al (2004) have indicated five

different applications of data hiding techniques:

1- Copyright Protection: can be obtained by inserting a robust watermark,

such as owner ID, signature, or logo, into a digital document and this


19

embedded mark should not be eliminated by any malicious attacks applied on

the document unless the protected media becomes corrupted and worthless.

2- Fingerprinting: aims to embed imperceptible and robust distinctive

signatures (e.g. the recipient’s name) into each copy of the document to allow

digital data owners or distributors to discover any collusion that may happen

after distribution.

3- Copy prevention and copy control: Copiers such as photocopy machines

and CD writers can be designed to prevent illegal copies or limit the number of

copies if they are designed to detect the presence of a watermark already

embedded in digital or paper documents and, then, either to forbid or allow

copying depending on the system requirement.

4- Authentication: is the detection of malicious manipulations that may possibly

be applied on the document (e.g. changing the holder’s name on a passport or

a number in a cheque). Authentication can be achieved by embedding

information about the original document into the document itself, without

affecting the quality of the host. In the verification process, the document can

be considered as genuine if and only if the extracted watermark and the

document information are alike. The research described in this thesis is aimed

at the authentication of digital binary document images.

5- Metadata binding: used to embed metadata information into an image for

purposes such as product advertisement in digital cameras.

Digital images can be verified by cryptographic authentication methods using

hash functions to determine whether they are authentic or not. However, the

protection of image integrity cannot be guaranteed following image decryption. Also,

hash functions are incapable of locating or quantizing changes that might have been

applied to these digital images. Therefore, all digital data such as document images,


20

with malicious or desirable changes, are not verified during the authentication

process in spite of the magnitude of the visual impact on the altered data (Fridrich,

2000). One of the suggested solutions, for recognising the regions that have been

tampered with maliciously, is the use of digital watermarking techniques (Fridrich,

1999a). Consequently, watermarking is used to protect data copyrights, to trace

illegitimate copies, and to find out sources of leaked information.

2.4 Digital Watermarks

Digital watermarking can be defined as an information hiding technique that

inserts signatures into digital content for the purpose of data ownership protection

and authentication. The inserted watermark must be imperceptible, secure, and

robust (Furht et al, 2005). In document communication, digital watermarking can be

used to assist a document recipient to recognize document owners and also to verify

the genuineness of documents (Furht et al, 2005).

2.4.1 Watermarking Applications

Digital watermarks can be embedded into multimedia files for different purposes

and applications. Furht et al (2005) classify the different applications of digital

watermarking as shown in Table (2.2).


21

Application Class Watermark purpose Application Scenarios

Protection of intellectual property rights

Conveys information about content ownership and intellectual property rights

• Copyright protection • Copy protection • Fingerprinting

Content verification

Ensure that the original multimedia content has not been altered and/or helps to determine the type and location of alternation

• Authentication • Integrity checking

Side-channel Information Represent side-channel used to carry additional information

• Broadcast monitoring • System Enhancement • Covert communications

Table 2.2 : Applications of Digital Watermarking (Furht et al, 2005)

2.4.2 Watermark Visibility

There are two different types of watermarks in terms of perceptibility. Visible

watermarks are usually presented as a corporate logo or visual text and this kind of

mark is used to protect the copyrights of digital media. Invisible watermarks cannot

be seen with the naked eye and thus extraction or detection methods are required to

take out or to check the existence of such watermarks (Kim & Afif, 2003; Kim & Afif,

2004).

The main aim of using visible watermarking is to warn users of digital media

against copying or misusing the copyrighted materials. These kinds of watermarks

are easy to implement and also easy to remove (Sun et al, 2001).

2.4.3. Watermarking System

Lan and Tawfik (1999) classify watermarking systems into three categories:

1- Private watermarking system: there are two types of this system and both

types require the original image; Type I is used to extract the embedded

watermark while Type II is to check the existence of the watermark.


22

2- Semi-Private watermarking system: detects the presence of the

watermark, as does Type II; however, it does not use the original image.

3- Public watermarking system: extracts the embedded watermark without

using the original image or the key. It is also called Blind Detection System.

2.4.4. Watermarks Robustness

Watermarks can be classified as robust or fragile according to their resistance

to attacks and each class is used for different purposes. Copyright and ownership

protection applications require using a robust watermark while fragile watermarks

are used for authentication systems because they are very sensitive and can be

removed with ease; consequently, when watermarked media are attacked,

watermark detectors may fail to detect the embedded signature and, then, fail to

validate the tampered files (Kim and Queiroz, 2004b).

Robust watermarks are used for copyright protection and this type of watermark

must not be removable by ordinary image processing tools such as image resizing

and compression. Fragile watermarks are sensitive to any manipulation applied on

the watermarked data and this kind of watermark is used to determine whether the

watermarked data have been tampered with or not (Kim & Afif, 2003; Kim & Afif,

2004).

Kim and Afif (2003) believe that only cryptography-based watermarking

techniques are reliable for authentication purposes while other watermarking

techniques cannot be robust.

The main difference between copyright protection watermarks and authentication

watermarks is that copyright protection watermarks are designed to be robust and

must survive not only all unintentional alternations but also any intentional attacks

that may be applied on the cover media. However, authentication watermarks should


23

be more sensitive in order to detect any alternations applied on the host media (Li et

al, 2003).

In Fridrich (1999a), authentication watermarking techniques are classified into

three main categories according to their robustness: fragile watermarks, semi-

fragile watermarks, and robust watermarks where each class has been designed

for particular purposes. Also, it is possible to combine two or more types of

watermark together to generate a hybrid watermark provided that the second

chosen watermark to be inserted (usually fragile) does not exterminate the

previously embedded watermark.

2.4.4.1. Fragile Watermarks

They provide a high level of vulnerability and can be easily corrupted by any

possible modification that might be applied on a watermarked image such as

changing a pixel value. One of the most well-known primitive fragile watermarking

techniques is based on setting the Least Significant Bits (LSB) of the image pixels

(Fridrich, 1999a).

2.4.4.2. Semi-Fragile Watermarks

The second category of watermark is semi-fragile. It is less sensitive than the

first type and can be used to gauge the corruption in the attacked images. It is

necessary for it to be robust against ordinary image processing operations such as

JPEG compression and brightness adjustment; simultaneously, it must be vulnerable

to malicious perceptible attacks such as cropping or adding additional features to the

watermarked images. Semi-fragile watermarking techniques use either the frequency

or spatial domain depending on the requirements of the system to be designed. A

threshold can be set to adjust the fragility of the designed technique (Fridrich,

1999a).


24

2.4.4.3. Robust Watermarks

They must survive regardless of applied attacks, whether they be malicious or

conventional, unless the watermarked image is visibly damaged to the extent it has

been made worthless. Therefore, this kind of watermark is mainly proposed for

ownership and copyright protection. The majority of robust watermarking techniques

are based on frequency transforms such as Discrete Cosine and Wavelet transforms

(Fridrich, 1999a).

2.4.5 Imperceptibility vs Robustness

Kwok et al. (2000) point out that all visible watermarking techniques proposed in

the literature are easy to remove or corrupt while all invisible watermarks are robust.

They also believe that all text watermarks have to be invisible and fragile, audio

watermarks must be inaudible and robust, video watermarks must be invisible and

robust, and image watermarks have to be either visible-fragile or invisible-robust.

However, these assumptions and categorizations cannot be strictly true because

different watermarking systems have different requirements. Embedding a

watermark in an image’s LSBs is a simple example of an invisible-fragile image

watermark; therefore, it is not necessary for it to be visible-fragile or invisible-

robust.

Nowadays, a contemporary category of image authentication techniques is being

investigated called self-embedding watermarking which aims to identify and

restore corrupted parts in attacked images as it stores the image content into the

image itself. This type requires high data embedding capacity which makes it difficult

to conserve the quality of the watermarked image. Furthermore, it seems to be

unlikely to provide a robust self-embedding scheme capable of surviving all available

image processing applications such as lossy compression (Fridrich, 1999a).


25

In content-based data hiding techniques, a message authentication code

(MAC) or a digital signature (DS) must be generated from the original image and,

then, embedded into the same image to produce the watermarked image. Inserting

the MAC/DS to the original image causes some changes to the host image; therefore

a subsequent derived MAC/DS from the watermarked image will not be identical to

the inserted MAC/DS. As a result, the verification process will not be successful (Kim

& Afif, 2003; Kim & Afif, 2004).

For that reason, least significant bits (LSB), in colour and greyscale images,

must be cleared before generating the original MAC/DS and, then, used to hold the

code or the signature. However, this cannot be applied on binary images because

each pixel is represented by one bit and any modification to the binary grid will cause

salt-and-pepper noise and the watermark will be invalid (Kim & Afif, 2003; Kim &

Afif, 2004).

2.4.6 Attacks on Watermarks

In authentication and forgery exploration, there are three main causes of

extraction failure during the detection process. Firstly, it may be caused by

intentional alteration to the meaning of a sentence or value of a number and, in this

case, the detector must discover this kind of forgery and inform the recipient that

the document has been counterfeited. The second possible attack could occur by

image processing operations such as rotation, skewing, and compression, which do

not change the content of documents, and detectors need to ignore this sort of

regular change. Also, detectors may fail to validate document images if they are

basically not watermarked and belong to another owner.

Therefore, it is preferable to embed yet another watermark which is robust to

malicious attacks for the purpose of identifying legitimate owners. A digital


26

watermarking technique that inserts two watermarks is called two-level

watermarking. Watermarks must not be integrated into file headers because the

conversion of a file format, (for example from .BMP to .GIF), and D/A conversion

such as a print/scan attack eradicate watermarks embedded in headers (Furht et al,

2005). Imperceptible data embedded in digital images for the purpose of detecting

potential changes are called authentication watermarks (Kim & Afif, 2003; Kim & Afif,

2004).

2.4.7 Image Watermarks Domains

Watermarks can be embedded in two different domains into digital images

(Hsieh et al, 2004):

1- Spatial domain: a simple approach where the watermark stream is directly

embedded into the host by modifying the pixel values of the image. The most

popular approach is altering LSBs in colour and greyscale images or flipping

pixels in binary images. Watermarks embedded in the spatial domain are

fragile and it is easy to eliminate inserted data by applying one of the image

processing operations such as JPEG compression or smoothing.

2- Frequency domain: Watermarks are embedded in the most significant

components of the host image after applying one of the frequency-domain

transforms such as Discrete Cosine Transform (DCT) and Fast Fourier

Transform (FFT) on the image. Hsieh, M. & Tseng, D. (2004) believe that

frequency domain watermarking techniques are not only more robust to

common attacks, especially image compression, than spatial domain

techniques but also provide larger data hiding capacity.


27

2.4.8 Watermark Properties

A reliable image watermarking technique must have the following features

(Hsieh et al, 2004):

1. Imperceptibility: The inserted watermark should be invisible to the naked

eye and the watermarked image must appear to be an identical copy of the

original.

2. Robustness: Watermarks must survive image processing operations and

resist other deliberate attacks that could be applied to the watermark image

in order to eliminate the inserted watermarks.

3. Unambiguousness: Embedded marks must contain sufficient information

about the lawful owner of the protected image to prove the ownership.

4. Security: A strong cryptography method can be used to achieve a high level

of security to prevent illegal users from accessing hidden information. Only

authorized private/public key holders are permitted to decrypt watermarks.

2.4.9 Watermarks for Document Images

In Kim and Oh (2004), watermarks can be inserted into document images at

three levels. The first one is the character-level which adjusts characters, words or

lines by shifting them upward, downward, left or right. The second is Feature-level

which alters stroke width or serif shape of letters or modifies the font formatting of

the text such as colours, sizes, or form. Pixel-level is the third which is not robust

even against simple attacks but is suitable for embedding large amounts of data.

In addition, replacing some words with synonyms, provided that the meaning is

equivalent to the original, is another way to watermark plain text but this method is

not applicable on some texts, for instance, poems (Arnold, 2003).


28

According to Kim and Queiroz (2004b), there are 3 approaches to embedding data

in digital binary or halftone images:

1- Pixel-wise: This directly alters the pixels values. Consequently, it may cause

salt-and-pepper noise if applied on binary document images but is more

applicable for halftone images.

2- Component-wise: This modifies certain features of the image content and

cannot be applied to all images as its reliability depends on the details of the

image to be watermarked.

3- Block-wise: This splits the images into small blocks (e.g. 8x8) and then embeds

watermark bits into each block depending on its features.

Other data hiding techniques for document images have been proposed. Chen et

al (2005) have classified these techniques with regard to the embedding process as

follows:

1- Text line, word and character shifting: can be applied on formatted text

documents by shifting lines, groups of words, or groups of characters slightly

to the left/right or up/down to encode a 0 or 1. Line shifting techniques

conspicuously show better robustness against D/A attacks such as print/scan

than other methods while character shifting techniques have a higher data

hiding capacity. However, different languages and fonts play a significant role

in robustness and capacity of word and character shifting methods.

2- Fixed partitioning into blocks: divides an image into small blocks with

fixed size and flips one pixel or more within selected blocks and without

creating artefacts depending on the characteristics of those chosen blocks.

The capacity of data hiding using this technique depends on the contents of

the document and it can be increased at the expense of the possible

perceptibility of the added data.


29

3- Boundary modifications: these can embed 5.69 bits of data in the 8-

connected boundary of each character or connected components as shown by

Mei et al (2001). A blind decoding method without using the original

document can be used to extract the inserted data.

4- Modifications of character features: this approach embeds data by

changing pre-extracted character features of the document. It offers

considerable resistance to D/A conversion but still needs more enhancements

in order to be robust against photocopying. Its data embedding capacity relies

on the number of characters in the document.

5- Modifications of run-length patterns: data can be embedded using this

method by shortening or lengthening the run-lengths of the black pixels

within a facsimile image.

6- Modifications of Half-tone images: different methods have been

implemented to watermark halftone images but those techniques cannot be

applied on document images.

2.4.10 Comparison between Binary Image Watermarking Techniques

Chen et al (2004) have compared the robustness against the aforementioned

attacks, capacity, benefits/drawbacks, and limitations of several watermarking

techniques for document images as shown in Table (2.3).


30

Techniques Robustness Benefits(+)/ Drawbacks(-) Capacity Limitations

Line Shifting high Low Formatted text only

Word Shifting medium Low/Medium Formatted text only

Bounding Box

Expansion medium - sensitive to document skewing Low/Medium Formatted text only

Character Spacing medium

+ can be applied to languages

with no clear cut word

boundaries

Low/Medium Formatted text only

Fixed Partitioning

Odd/Even pixels none

+ can be applied to binary

images in general High

Fixed Partitioning

percentage of

white/black pixels

Low/ medium


images in general

-image quality may be reduced

High

Fixed Partitioning

logical invariant none

+ embeds multiple bits within

each block

+ use of a secret key

High

Boundary modification none


images in general

+ direct control on image quality

High

Modification of

horizontal stroke width medium Low/Medium

Languages rich in

horizontal strokes only

Intensity modulations

of sub-character

regions

medium Medium Greyscale images of

scanned documents only

Run-length modification none -image quality may be reduced

Use two-dithering

matrices none High Half-tone image only

Embed data at pseudo-

random locations none High Half-tone image only

Modified ordered

dithering none High Half-tone image only

Modified error diffusion none High Half-tone image only

Table 2.3 : A Comparison between Several Watermarking Techniques for Document Images (Chen et al, 2004)


31

2.5 Methods in Literature

A number of researchers have shown interest in data hiding techniques for digital

images. Their ideas and experiments are reviewed in this section.

Several data hiding techniques have been proposed in the literature for colour,

greyscale, half-tone, and binary images. Date hiding methods for images will be

discussed in the next two sections. Colour and greyscale data hiding methods will be

presented in the first section. The second section presents data hiding methods for

binary and half-tone images. Other data hiding techniques are discussed in section

2.5.3.

2.5.1 Colour and Greyscale Image Data Hiding Methods

Data hiding techniques for colour and greyscale images, as proposed by different

researchers, are discussed in this section. Each pixel in a colour image is represented

by up to 3 bytes while a single byte is needed to represent a pixel in a greyscale

image. Spatial domain data hiding techniques are discussed and evaluated first.

Next, frequency domain watermarking techniques are shown. Hybrid data hiding

techniques are presented in the third part. In the final section, data hiding

techniques using different embedding strategies are discussed.

2.5.1.1 Spatial Domain Data Hiding Techniques

Yeung and Mintzer (1997) proposed an invisible watermarking technique, for 24-

bit and greyscale image verification, based on embedding a small size image (logo)

repeatedly into the LSBs of the host image (to be protected) which must have larger

coordinates to hold the watermark. In the verification process, the method uses a

verification key, which is previously created during the embedding operation, to


32

extract the embedded watermark and to recognise the parts of the image which have

been tampered with. Obviously, this invisible watermark is fragile and unable to

survive regular image processing operations as well as the D/A conversion because it

embeds the watermark in the LSBs of the host image and those bits can easily be

flipped from 0 to 1 or vice versa when a pixel value is modified. Also, this method

cannot be used for binary images because each pixel is represented in a single bit in

black & white images.

Fridrich et al (2000) proposed a fragile authentication watermarking technique for

images. The method aims to avoid the weaknesses of the scheme proposed by

Yeung-Mintzer (1997) by substituting the binary look-up tables of individual pixels

for an encryption map of local neighbours to prevent counterfeiters from estimating

both the key binary function and the logo used repeatedly in different images. Also,

it determines whether an attacker has pasted together sub-images from different

source images while retaining their locations in the attacked image (collage attack).

Fridrich and Coljan (1999) proposed a self embedding image watermarking

technique to retrieve the original contents after tampering. The technique is based

on hiding the DCT coefficients of an image inside the image itself.

In the watermark embedding process, the original image is divided into 8x8

blocks and a DCT transform and quantization are applied to each block. Then, the

first 11 low-frequency values, in zigzag order, of the quantized blocks will be

encoded into 64 bits and embedded in the LSBs of blocks in the original image

provided that the host block does not hold its quantized values.

In this case, if an important part of an image has been distorted, such as the

name of a licensed person in a certificate, it is possible to recover these items by

extracting them from an unaltered part of the same image. Two LSBs are also used

in order to enhance the data hiding capacity and the quality of hidden images.


33

On the other hand, it is not possible to reconstruct the tampered part if the

embedded code area has also been attacked. In addition, this technique is fragile

and any image processing operation which alters the LSBs of the watermarked image

can remove the watermark.

Wong (1998) introduced a public key watermarking method to verify the

authenticity of greyscale and colour images in the spatial domain. The method

basically inserts the watermark into the LSBs of greyscale images. In colour images,

the watermark can be embedded into the Red-Green-Blue (RGB) palette or into the

Luminance-Bandwidth-Chrominance (YUV) colour components. A binary image

having the same dimensions as the host image is used in this method as a

watermark.

In the embedding process, the LSBs of the original image X are flipped to zeros

to form X’. After that, both X’ and the binary watermark image B are divided into

blocks fixed in size. Each block belonging to X’ is then hashed with a cryptographic

hash function H. The Exclusive-OR (XOR) operation is then applied between each

output block of the hash function and its equivalent watermark block. The output

blocks are then encrypted with a private key K’ and the resulting binary grid is then

embedded in the LSBs of X’ to generate the watermarked image Y.

To extract the watermark, the LSBs of the watermarked image Z, which could

have been tampered with, are extracted first to form the binary grid G which is then

divided into blocks. Every block is then decrypted by the public key K to produce the

binary matrix U. The LSBs of the watermarked image are then set to zero to

generate Z’ which is also divided into blocks. A hash function is then applied on each

block to produce the 2D-matrix Q. The watermark O is then extracted by applying

the XOR operation between the hashed matrix Q and the decrypted matrix U.


34

If the extracted watermark is the same binary watermark inserted earlier, the

test image has not been altered. The inserted binary watermark image should be the

logo, name, or signature of the legal owner or originator of the image. The Message-

Digest algorithm-5 (MD5) was used as the hash function in this method while Rivest,

Shamir, & Adleman (RSA) algorithm was the chosen public key encryption algorithm.

This method embeds invisible fragile watermarks in greyscale and colour images

for the purpose of verifying image authenticity. It can detect any modifications

occurring in the watermarked image such as image resizing or changing pixel values.

However, this method is certainly not applicable on binary images and has a limited

range of applications because it is based on hiding in the LSBs.

Mohanty et al (1999) proposed a dual watermarking technique to protect the

copyright and ownership of colour and greyscale images. The method embeds two

watermarks (visible and invisible respectively) into the host image to create the dual

watermarked image. The visible watermark should be inserted first because it may

destroy the invisible watermark if inserted later.

To embed the visible watermark, both the original image (I) and the

watermarked image (W) are divided into blocks of the same size. The mean and

variance of each block are then computed and used to adjust the scaling and the

embedding factors for inserting the watermark image into the original.

The embedding process is applied in a block-wise manner and there are some

necessary restrictions in order to avoid serious distortion to the content of the

original image and degrading of its quality. In order to keep the original details of the

image, edge blocks must not be altered significantly. Blocks with high variance of

colours are subject to more modification than those with uniform intensity because

they are less sensitive to modifications.


35

Afterwards, the invisible watermark is randomly inserted into the visible

watermarked image in the spatial domain. The most significant bits (MSBs) of the

original image pixels are chosen first to hold the watermark binary sequence.

However, if this would cause a significant distortion that increases the SNR

significantly (a preset threshold is used), less significant bits are used instead to

carry the watermark. The extraction of the invisible watermark is required only if the

visible watermark was removed or to detect unintentional modifications to the dual

watermarked image.

This technique is applicable only on colour and greyscale images and cannot be

used to protect black and white images because it is based on hiding the watermarks

in the colour intensities or the bit-plane of the original.

Li et al. (2003) proposed a content-based watermarking scheme to verify the

integrity and authenticity of images. The method simply divides the original image

into fixed-size blocks, extracts features from the 7 most significant bits (MSBs) of

each block, and represents each pixel in a single bit watermark. The watermark

blocks are then blended by replacing the right half of each block with the right half of

its neighbour in a zigzag path in order to make the scheme able to locate any

modification to the watermarked image that may occur later.

Afterwards, the blended blocks are encrypted with the private/public RSA

algorithm to increase the security of the method. Therefore, if an attacker uses the

same watermarking method, with a different RSA key, to create a false watermark

and then resends this image to the intended recipient, the received image will fail to

be unauthenticated by the verification system when the RSA public key is used. The

encrypted binary blocks are then embedded in the least significant bits (LSBs) of the

image to form the watermarked image.


36

In the verification process, the LSBs of the watermarked image are extracted,

divided into blocks, and encrypted with the public RSA key. Next, the watermarked

blocks are recombined. The extraction of image features from the MSBs as done

during embedding is also required. Finally, the two extracted watermarks are

compared against each other. If they are identical, the image is authentic otherwise

the image has been tampered with.

The method is simple and there is no need for sending prior information about

the original, such as image size, to the verifier. Only the public key is required during

the verification process. Adding objects, using a wrong RSA key, equalizing the

histogram, scaling, and cropping are five different attacks applied on an image in Li’s

experiment and the method has not confirmed the authenticity of the altered images.

Also unwatermarked images have been tested and rejected by the verifier. Only the

intact watermarked image was verified.

However, this method has some disadvantages as it is applicable only on colour

and greyscale images, so, binary images cannot be watermarked by this method.

Also, it is very sensitive, in the sense that it cannot verify the integrity of images if

ordinary image processing tools, such as compression and brightness/contrast

adjustment, are applied to the watermarked image despite the fact that the visual

details of the image are not affected. Therefore, it certainly fails to verify images

after D/A conversion.

2.5.1.2 Frequency Domain Data Hiding Techniques The Anti-Forgery Identification Document (AFID) is a technique proposed by

Chow et al (1993) and designed to protect IDs against forgery. AFID uses a

private/public key encryption method and barcodes to generate and verify

identification cards.


37

The construction process of an identification card consists of a number of stages.

First of all, an asymmetric RSA encryption algorithm is used to generate two 592-bit

encryption keys, a private key used in creation and a public key used during

verification. Some information such as the card holder’s name and the expiry date of

each card produced must be stored in a card database file to be used for

authentication.

The “Picture Descriptor” is extracted from the card holder personal photograph

which usually appears in one of the corners of the card. The picture descriptor is

generated by dividing the photograph into parts and calculating the gray level of

each part. Next, the gray level of each part is encoded into three different numbers

using two pre-determined thresholds. Any value less than the first threshold is

considered as black, any gray level higher than the second threshold is set to be

white, while other values between these two thresholds are treated as gray. The

vector is then encrypted with the private key to produce the picture descriptor.

Afterwards, the picture descriptor is represented in a two-dimensional barcode

containing 80 bytes of data (20 bits of height x 32 bits of width); the barcode is then

attached to the generated ID card.

For verification, a tested card must be scanned and the same descriptor vector

should be extracted from the photo. Also, the encrypted vector previously embedded

in the barcode must be extracted and decrypted using the public key given to the

verification station. The Euclidian distance between the two vector descriptors is then

calculated. The scanned ID card is determined to be a forgery only if the distance is

greater than a preselected threshold; otherwise, the authenticity of the ID is

certified. Therefore, attempts to replace an original photo with a forged one will be

unsuccessful. The biographical information printed on the surface of the card can

also be a counterfeiter’s target; though, any inconsistency between the information


38

written on the card and the data extracted from the security seal will result in

rejecting the authenticity of the card.

The AFID technique depends on the security level of the public key; therefore, a

secure cryptography method must be carefully chosen to safeguard the secret

information in the seal. Also, the limited capacity of the barcode, which holds the

secret information, may decrease the reliability level of this technique.

Fridrich (1998a, 1998b) proposed an invisible robust watermarking method to

detect malicious changes in digital colour images. It divides the original image into

medium size blocks (e.g. 64x64) and inserts spread spectrum signals of the image

contents into the middle frequencies of the DCT coefficients of those blocks. The

embedding procedure of this method can be used as copyright marks for images in

digital cameras as it requires only small memory size and a short computational

time. It can detect some undesirable alterations such as cropping and replacements

while regular image processing operations, for instance a slight increase/decrease in

image sharpness/brightness, are undetected.

Nevertheless, it cannot be used to protect binary images because it uses the

frequency domain to hide watermarks. Also, its data hiding capacity and its

robustness are dependent on the characteristic of the host blocks where blocks with

a plain area do not have as large as embedding capacity to retain the watermark as

do textured blocks.

Lan and Tawfik (1999) have proposed self-embedding watermarking technique in

order to detect tampering and to retrieve the corrupted areas in colour images. It

embeds some important parts, expected to be attacked, of the image into the DCT

coefficients of the image itself. The data hiding capacity of their algorithm is about 1

watermark bit per 167 bits of the host image. However, the embedding process

cannot determine the visual distortion; therefore, users of this technique are


39

supposed to stop embedding more data once there is perceptible artefacts. Also, this

technique cannot be applied on binary and halftone images because it is based on

the DCT.

Self-Authentication-and–Recovery Image Watermarking System (SARI) is a semi-

fragile technique designed by Lin and Chang (2001) and used to authenticate

images. SARI can identify malicious attacks, discover ruined areas and almost

recover original regions by extracting hidden information about the original image

from the attacked watermarked image itself.

In the embedding process, this system inserts imperceptible watermark in digital

images, while in authentication and recovery it detects many malicious attacks

excluding some desired operations such as JPEG lossy compression. Also, SARI can

locate the modified parts of the watermarked image and may be able to reconstruct

the affected areas. The free trial version of SARI software and its authenticator are

available online at: http://www.ee.columbia.edu/sari/auth.html. The watermark in

this technique seems to be designed to survive JPEG compression attack provided

that the quality of the compressed image is not visually corrupted.

Kim and Oh (2004) proposed a digital watermarking algorithm for text-based

images that embeds the watermark in edge-direction histograms of the greyscale

document image. Their method initially divides the image into sub-images of words,

lines or fixed-size blocks and afterwards constructs the edge direction histograms

quantized into 16 levels for each block. The method has been applied to 3 text-

images written in English, Korean and Chinese languages. Subsequently, it

designates one block or more as “mother block” while other blocks are considered as

“child blocks”. One bit of the watermark is inserted into each child block by enlarging

or shortening the histogram directions of the chosen block depending on the binary

http://www.ee.columbia.edu/sari/auth.html�


40

value of the watermark bit. However, the mother block must not be modified. The

peak signal-to-noise ratio PSNR of this method approximately is 13.5 after insertion.

The watermarked images were tested by applying various attacks such as

rotation, noise addition, column deletion, blurring, binarization and sharpening and

showed plausible robustness. However, it may fail to extract the inserted watermark

after a print/scan operation. Kim and Oh (2004) believe that their algorithm is

functional in some practical areas, for instance, digital libraries.

Hsieh and Tseng (2004) proposed a Discrete Wavelet Transform (DWT) based

watermarking technique to embed a greyscale image into another image

proportionally larger in size.

In embedding, a 3-level DWT sub-band decomposition is applied on both the

small greyscale image (watermark) and the large image (host). Then, the DWT sub-

band coefficients of the watermark are inserted into the high context energy DWT

coefficients of the host to achieve a high level of robustness as well as

imperceptibility. The Inverse Discrete Wavelet Transform (IDWT) is then applied on

the modified host sub-bands to reconstruct the watermarked image.

For detection, a similar 3-level DWT is applied on the watermarked image to

extract the watermark sub-band coefficients using an extraction key. Afterwards, the

IDWT of the extracted sub-band must be computed to reshape the extracted

watermark. The existence of the original watermark is required to assess the quality

of the extracted image by using the peak signal-to-noise ratio (PSNR).

Their technique has been tested and has shown a considerable level of

robustness to JPEG compression, sharpening, blurring, and smoothing operations.

However, this technique is not applicable on binary images and is not robust to print-

and-scan attacks.


41

Sharkas et al (2005) also proposed a DWT-based watermarking technique to

protect the copyright and to prove the ownership of digital colour images. To insert a

watermark, a 2-level DWT is applied on the original image. Then, the watermark

image is embedded in the horizontal, vertical, and diagonal wavelet coefficients of a

host image which must be larger in size. Inverse DWT is required to produce the

watermarked image.

In detection, a 2-level DWT is applied, as in the embedding, and a key is needed

to extract the watermark from the modified parts of the watermarked image is DWT

coefficients.

Compression, low pass filtering, salt and pepper noise, and luminance changes

were the four different image processing operations applied on the watermarked

image in order to test the robustness of this method and the watermark was

successfully extracted from the attacked images. However, this method cannot

protect binary images as it is based on hiding large amount of data in the frequency

domain of colour images. Also, it could fail to prove the ownership of colour images

after a print/scan attack.

Han et al (2005) proposed a new watermarking algorithm for image copyright

protection based on Wavelet Transform. The technique embeds a 2D Data-Matrix

barcode as a watermark into the high and middle frequency subbands of the wavelet

coefficients to conserve the quality of the image and to avoid any visible artefacts

caused during embedding. IDWT is applied after the insertion process to construct

the watermarked image.

The method shows satisfactory robustness against JPEG and The Set Partitioning

in Hierarchical Trees (SPIHT) where 90% of the watermarks embedded in test

images were successfully extracted after compression. The 2D Data-Matrix barcode

is chosen because of the fact that the embedded barcode watermark can be


42

reconstructed even if up to 60% of the barcode is damaged provided that a Reed-

Solomon error correction code is used. More details about the 2D Data-Matrix

barcode are given in chapter 3.

However, this algorithm is unsuitable for protecting binary images because it is

based on the Wavelet transform. More investigation is required to make this method

applicable to moving photos and audio files. Also, it has not been tested for other

kind of attacks.

Sutcu et al (2007) proposed a wavelet transform based method to detect

tampering in colour and greyscale images. The method can identify

sharpness/brightness adjustments and region replacements applied to images by

counterfeiters. The regularity of the image must be estimated by applying an edge

detection algorithm first to find out the locations of edges in the image. After that, a

4-level undecimated discrete wavelet transform (UDWT) of each row and column of

the image is computed separately. The UDWT is the discretized version of the

continuous wavelet transform. Then, the row-based and the column-based linear

curve fittings of the highest amplitude values of the UDWT coefficients are measured.

The average of these two fittings represents the final sharpness/blurriness measure.

In detection, if this value increases or decreases significantly, the test image will be

considered a forgery.

However, this method works only on colour and greyscale images and cannot be

used for binary images. Also it can detect sharpness/brightness adjustments and

copy/paste attacks but it may fail to detect counterfeiting if other image processing

tools are used.


43

2.5.1.3 Hybrid Data Hiding Techniques

Fridrich (1999b) presented a hybrid watermarking technique that embeds two

watermarks (robust and fragile) to detect tampering in colour images. A robust

watermark is firstly inserted in the DCT coefficients of the transformed image and it

is used to detect obvious unwanted modifications such as part replacement of the

watermarked images with fractions of other images. Then, a fragile watermark is

embedded in the LSBs of the image grid without causing any significant distortion to

the previously added watermark. The second embedded watermark discovers all

simple image processing operations that could be applied to the watermarked image.

In detection, the fragile watermark can be extracted directly from the LSBs while

IDCT is required to extract the robust watermark from the DCT coefficients.

However, this technique needs more improvements to be robust against geometrical

operations such as cropping and resizing. Also, it cannot be applied on binary images

because it is based on utilizing the LSBs and the DCT coefficients of the image.

Tsai et al (2007) proposed another hybrid feature-based invisible image

watermarking technique for copyright protection and integrity verification. The

Hessian-Affine feature detector is used to locate the robust feature spots in the

image. Next, the image regions are divided into two groups, one for the copyright

signature and another for the verification code. Due to their resistance capability to

image processing operations, the detected robust regions are chosen to embrace the

copyright signature while the other parts of the image are used to carry a verification

fragile watermark in their LSBs. The two groups are then combined to form the

watermarked image. A secret key and a hash function are used during embedding

and extraction. The embedding and detection schemes are illustrated in figures (2.1)

and (2.2) respectively.


44

Original Image

Hessian-Affine

Detector

Region Selection

Copyright insertion

Authentic insertion

Watermarked image

Watermarked Image

Hessian-Affine

Detector

Region Selection

Copyright detection

Authentic detection

Ownership verification

Content Authentication

Figure 2.1 : The Block Diagram of Watermark Embedding Scheme (Tsai et al, 2007)

Figure 2.2 : The Block Diagram of Watermark Detection Scheme (Tsai et al, 2007)

Once again, this algorithm is not suitable for binary images because it utilizes

the LSBs of the host image to hide the watermarks. Moreover, the watermark can be

easily corrupted by the D/A conversion even if the watermarked image is not

tampered with. The time complexity of this algorithm is high due to the repetitive

process of the Hessian-Affine detector.

2.5.1.4 Other Data Hiding Techniques

A content-based digital signature method for image authentication was proposed

by Schneider et al. (1996). Continuous authenticity measures are used in the

verification process of this method to create more flexibility to confirm the

authenticity of images that have been manipulated with JPEG compression or noise

reduction tools without affecting the visual objects of the watermarked image.

Therefore, the tested image could be completely authentic, partially authentic, or

unauthentic. A threshold needs to be set in order to obtain optimal verification

results and this should be variable depending on the requirements of the system.


45

To generate the signature, the original image is firstly divided into blocks and

the intensity histogram of each block is then computed. After that, the Euclidean

distances between intensity histograms are calculated. Subsequently, the data size is

then reduced by hashing the histograms with a hash function fh. Afterwards, the

hashed data is encrypted with a secret cryptographic key Kpr to produce the

signature S.

In verification, the same steps done previously in the signature creation stage,

excluding the encryption, must be applied on the test image. The signature S is then

decrypted by the public key Kpu and compared with the hashed data of the test

image using a threshold tau. The test image is authentic only if the difference

between the values of the feature vectors is smaller than tau.

The threshold can be used to reduce the sensitivity of the verifier and

experiments are needed to set the optimal value of tau. For example, to make this

method robust to JPEG compression, the highest difference between vector values of

both the original and compressed image must be computed. The threshold must

have a value equal to or higher than the maximum difference. However, the

threshold must be set to zero if the hash function is used.

The main drawback of this method is that the extracted signature S is not

embedded into the image itself and it must be sent separately to the verifier which

reduces the security level of this method. Also, it requires a large database to save

the encrypted histograms of images. In addition, this method cannot be used to

verify binary images because it is based on computing the image histogram and this

would just be a count of the frequency of 1s and 0s.


46

2.5.2 Binary and Half-tone Image Data Hiding Methods

Despite the digital revolution, paper documents are still required in business and

offices and these documents are normally printed in black and white (Zhu et al,

2003). Also, binary document images have been widely-used in daily life as

immigration papers, birth certificate, bank cheques, books, deeds, music scores, or

other official documents. However, counterfeiters can exploit cheap scanners,

printers, photocopiers and other office devices to create forged copies to attain their

malicious objectives (Chen et al, 2004).

There is more flexibility and capacity to embed watermarks in colour and

greyscale images because each pixel in those types of images is represented by

more than one bit, for instance each pixel in true colour images is represented by 24

bits. Therefore, the majority of data hiding techniques proposed in the literature are

suitable for colour and greyscale images but those techniques are not applicable to

binary images that have an obvious division between foreground (texts) and

background regions. This lack of binary image data hiding methods is a result of the

dual nature of binary images where each pixel is represented by a single bit, 0 for

black and 1 for white. Consequently, any alternation in binary images typically

causes perceptible artefacts and visual irregularities such as salt and pepper noise.

(Furht et al, 2005; Yang and Kot, 2004; Lu et al, 2003; Mei et al, 2001; Low et al,

1998; Arnold, 2003; Kim and Oh, 2004).

Half-tone images are black and white images which give the impression of being

greyscale images if seen from a distance. Some of the images shown in newspapers,

fax documents, and books are half-tone images. Dispersed-dots and Clustered-dots

are two different types of half-tone images. However, they differ in visual quality

where clustered-dots half-tone images have poorer quality than dispersed-dot

images. The process of converting greyscale images to binary images is called half-


47

toning (Kim & Afif, 2003; Kim & Afif, 2004). The techniques proposed to hide data in

binary and half-tone images will be reviewed in this part of the thesis.

The major difficulty in implementing a reliable authentication technique for

printed documents is the trade-off between the robustness of the inserted

authentication label and the ability to avoid false verification due to the print/scan

operation causing noise in the scanned document which results in false-negative

detection (Sun et al, 2001).

Low et al (1998) proposed a method to watermark document images by shifting

words in the original document slightly left or right or by shifting a whole line up or

down insignificantly as well as to identify the embedded watermark in those images

by using centroid detection. The watermark in this method can be easily removed by

retyping the text or using scanners with Optical Pattern Recognition (OCR).

In their detection process, the horizontal profile of lines must be compiled to

detect line shifts, or alternatively, it is essential to compute the vertical profile in

word shifts, and afterwards a comparison between the centroids of profiles of the

original and the watermarked text images should be made to obtain watermarks.

The experimental results in the presence of noise caused by printing,

photocopying and rescanning show that the performance of centroid detection for

line shifts is considerably better than that for word shifts.

Mei et al (2001) proposed a method to watermark binary document images by

inserting data in the 8-connected boundary of each letter. Data can be embedded, in

this technique, by matching and replacing a selected set of connected boundaries

with a predefined pattern stored in a lookup table to indicate 0 or 1. The data can be

easily extracted by using the same method without referring to the original

document image. The substantial level of imperceptibility has been achieved as data


48

are inserted in non-smooth border edges. The capacity of data hiding depends on the

resolution of the image and it can be increased by using the inner boundaries in

addition to the outer lines. It can be useful for adding short labels to document

images.

Lu et al (2003) proposed a watermarking technique using the Distance Reciprocal

Distortion Measure (DRDM) which is used to select the pixels to hold a watermark

with the lowest visual distortion in the watermarked image.

The method also uses a 2D shifting technique and odd-even scheme in the

embedding process. The 2D shifting is essential to make tampering detectable in

extraction. The experiments show a high rate of imperceptibility and easy extraction

of the watermark.

Zhu et al (2003) proposed a novel print signature technique to embed a unique

signature in each printed copy alongside embedding information about document

contents. The method includes registration and authentication procedures.

In registration, the digital signature is extracted from certain features in the

document as well as a unique document identifier which is derived from critical

information about the document to be protected. Then, both signature and identifier

are added to a blank area of the document as a barcode; In addition, some

supplementary landmarks must also be printed for the purpose of alignment. Only

documents which contain inserted signature and identifier can be circulated.

The authentication procedure is used to determine whether the printed document

has been tampered with or not. It extracts the signature and the identifier from the

scanned document as done in the registration process and then compares them with

the information attached to the same scanned document as a barcode. The

document is accepted as genuine if the extracted features are alike while it is

considered as a forgery if not.


49

This technique might be functional in online billing, voting ballot, and e-ticket

systems and it has been tested by using two HP LaserJet printers (8100 and 4600) to

print out documents on plain, translucent, card, and coloured papers. It has

successfully rejected all the 50 forged documents. On the other hand, some original

documents have not been accepted. Another weakness is that photocopied

documents have not been tested due to the poor quality of those documents. In

addition, more investigation is required to increase the robustness of the method

against photocopying and other acceptable modifications.

Data Hiding by Self Toggling (DHST) is a simple data hiding technique suitable for

half-tone images. It is based on creating an unduplicated set of pseudo-random

numbers representing random locations in the image (I), and then replacing the

value of this location in the image I with a bit of the watermark. The probability of

changing a bit from 0 to 1 and vice versa is 50%. In extraction, the pseudo random

numbers are needed to extract the embedded message. Salt-and-pepper noise will

result when DHST is used (Kim & Afif, 2003; Kim & Afif, 2004).

Authentication Watermarking by Self Toggling (AWST) is a cryptography-based

authentication watermarking technique for dispersed-dot halftone images proposed

by Kim and Afif (2003). The technique randomly clears a number of bits in a binary

image B using a pseudo-random number generator L to create B*; it then computes

the signature S using the equation below:

S= E (Ĥ) where Ĥ= H (B*) A, where H is a cryptographically secure hash

function, A is the logo image, and E is an encryption method. E could be either a

secret (symmetric) or public key (asymmetric) ciphering method. The obtained

signature S is then embedded into the set of pixels L to generate the watermarked

image B’.


50

The verification process uses the pseudo random number generator L to extract

the previously embedded signature S from the watermarked image X’ which could be

tampered with. Then, it clears the pixels in L to produce X*, and to compute a new

signature C using the equation C= H (X*) D(S). Where D(S) is the decryption of S

and H is the same secure hash function used in the embedding process. After that, C

is compared with the logo A. If they are equivalent, the image is authentic;

otherwise, it is forged.

However, the size of the host image has to be much larger than the size of the

logo image; otherwise, the watermarked image would be corrupted. Also, salt and

pepper noise could occur if this technique is applied on pure binary images, such as

document images with plain large black or white areas. In addition, AWST uses a

fragile watermarking system; therefore, it is very sensitive to any change

(deliberately or mistakenly) applied to the watermarked image. As a result, it is

wholly inappropriate for authenticating images after D/A conversion as the

embedded watermark is eliminated after such an attack. In addition, AWST can

detect changes in images but it is incapable of locating these alterations.

Kim and Queiroz (2004a) have proposed a new secure authentication

watermarking techniques for binary images. It uses secret-key pr public/private key

cryptography where only generators of document images can embed watermarks

while recipients can verify the image integrity by using the public key. It also uses a

fragile watermarking technique with the intention of detecting any perceptible

malicious alternations.

The embedding process is based on dividing a binary image into two parts with

different sizes; the authentication signature is computed from the first part (large in

size) which while the second (small) is used to carry this calculated signature. A


51

signature S is derived by applying a hash function on the first part, then encrypting

the results of the hash using a private key.

Undesired alternations to the first part can be identified by this method but it is

incapable of detecting possible changes in the second parts. Practically, it can be

used in document image communication via the Internet and further investigation is

required to adapt this technique to D/A conversion.

Kim and Queiroz (2004b) proposed a cryptographic authentication watermarking

technique for binary images where the original image Z is shuffled randomly with a

seed which is used as a secret key in private-key scheme or overtly distributed in

public/private-key scheme. Then, the shuffled image Ź is divided into two parts Ź1

which holds the signature and Ź2 where the signature comes from. After that, a

secure hash function H should be applied on Ź2 and the result encrypted to generate

the signature S=Ek(H(Ź2)). Finally S should be inserted in Ź1 by chopping Ź1 into

blocks (e.g. 8x8) and finding the lowest visually significant pixel in each block to be

altered to encode a single bit depending on the number of white pixels within each

block, where even numbers refer to 0 and odds are 1’s.

To verify whether the document image is authentic or not , first of all , the

secret/public key should be used as the seed of the random number generator to

shuffle the watermarked images Z̀ . Then, the shuffled image Ź̀ is divided into two

parts Ź̀1 and Ź̀2 as done in the embedding stage. Afterwards, a comparison between

the computed signature from Ź̀ 2 where S2=Ek(H(Ź̀2)) and the extracted signature

S1 from Ź̀1 should be made. The document image is genuine provided that S1 and

S2 are identical; otherwise, the image has been counterfeited or the key is incorrect.

Flipping a single bit in the whole image can be detected by this method.

However, it fails if two pixels within the same block have swapped their values.

Another point is that the data hiding capacity of this technique depends on the


52

content of the image. Further investigation is still required to adapt this technique to

D/A conversion.

Yang and Kot (2004) proposed a watermarking method to attest the authenticity

of binary image documents. The watermark in this algorithm is generated by a Gold-

like sequence whilst the embedding process takes the smoothness and topography of

the document image into consideration to eliminate the noise and keep the

connectivity in the watermarked document. The blocks of 5x5 neighbourhood pixels

to be flipped are chosen according to pre-defined patterns. In the authentication

process, the original image and the secret key are required to verify the legitimacy of

the text image.

This technique preserves the quality of image after the embedding process and it

could be useful for real time verification as it has short computational time. On the

other hand, the quantity of inserted data is limited and depends on the nature of

host documents. Also it is a non-blind watermarking technique and needs further

investigation to make it blind.

Alattar (2004) proposed a method of fingerprinting text documents for the

purpose of discovering any collusion done by an authorized employee by adding a

unique ID, which refers to each employee, into each given copy. This technique is

appropriate for left, right or centre aligned texts, justified paragraphs, and irregular

line spacing. The embedding process in this technique is based on shifting words or

lines of the texts slightly. It uses a spread spectrum technique to solve the issue of

uneven distances and BCH (Bose-Chaudhuri-Hocquenghem) error correction code to

reduce the noise caused by Digital to Analogue conversion. The original image is not

required during detection where watermarks can be extracted by computing the

distances between lines and words in the text. This algorithm presents a significant

rate of robustness but further research is required to protect inserted data against a

print-and-scan attack.


53

Xiaoyi et al. (2004) proposed a steganalytic technique to determine whether or

not a binary image contains a secret message. The technique is simple and tests

document images by calculating statistics U of an image I; then, embeds a test

message into the image I to obtain a stego Image I’. After that, the same statistics

U’ should be computed from I’. The block coarseness statistical analysis is used in

this technique to calculate U and U’. Initially, the binary image should be divided into

blocks; then, the edge irregularity of each block can be computed by using the block

coarseness. The Rate R = U’/U is then computed. The image I has already been

embedded with a hidden message if the value of R is approximately 1; but if R is

relatively greater than 1 by a predetermined threshold T, the Image I is plain and

does not contain a secret message. The technique has been tested by checking the

existence of secret messages embedded in binary photos using only two known data

hiding techniques. However, it may fail to detect the availability of embedded data if

other hiding techniques were used. Also, the detection might not succeed after print

and scan attack because objects in binary images could become thicker or thinner

and significant changes may occur to the coarseness of edges.

Puhan and Ho (2005) suggested a watermarking algorithm to authenticate

binary document images. To embed a watermark, the method initially computes the

curvature-weighted distance difference (CWDD) measure for contour pixels in the

original document image and then picks the pixels with estimated distortion values

less than a pre-defined threshold T. Those selected pixels are known as suitable

pixels while the rest of the pixels are non-suitable to hold the watermark. After that,

the suitable pixels are scanned in a sequential order left-to-right and top-to-bottom

to select the reversible pixels. There must be only a single reversible pixel in the

centre of an MxM block in the original image and this pixel should not affect the

condition of the other pixels in this block after inserting a watermark bit in it. These


54

criteria must be available in those chosen pixels to guarantee a similar scan order

during detection. Therefore, only the reversible pixels are chosen from the entire set

of suitable pixels to carry the watermark. Flipping the reversible pixels from 0 to 1 or

vice versa reduces the visual noise in the watermarked images to maintain the

imperceptibility of the method. The N irreversible pixels are then separated into two

sets SR and SM. SM is then hashed by an MD5 hash function to generate the Hashed

Message Authentication Code (HMAC). SR is used to carry the authentication

message extracted from SM. A secret key is used to embed SM pixels into SR.

In verification, the same procedures as in the embedding process must be done

to the test image to extract 2 subsets SR’ and SM’ and a comparison between pixel

values of those two set is then made. The test image is authentic only if SR’ and SM’

are equal. Otherwise, the document image is determined to be forged.

Document images transmitted or modified digitally can be verified by Puhan and

Ho’s method. However, this method is sensitive and it cannot verify hardcopy

documents after converting them to digital form. The method has limited embedding

capacity because it selects a particular set of pixels from the original image to hold.

Fridrich et al (2005) proposed the wet paper coding (WPC) method which is used

for secret communication and to solve the “Writing on Wet Paper” scenario in

steganography. The above-mentioned scenario can be summarized as follows:

“Imagine that the cover object x is an image that was exposed to rain and

the embedder can only slightly modify the dry spots of x but not the wet spots.

During transmission, the marked image y dries out and thus the detector does

not know which pixels were used by the embedder for data hiding. The task of

wet paper coding is to enable both parties to exchange secret messages under

the above scenario. The problem of data embedding in binary images fits this


55

“writing on wet paper” paradigm quite well, with flippables viewed as dry spots

and non-flippables as wet spots” (Gou and Wu, 2007, pp. 279).

The modification of flippable bits from 0 to 1 or vice versa during encoding may

change the flippability status of the neighbouring pixels which causes wrong

extraction at the recipient’s side. k flippable pixels must be found in the original

image X which has n pixels to embed the q bits of the secret message M into X to

form the marked image Y. A pseudo random binary matrix D with the size (n x q) is

then generated by using a secret key where M=DY. The matrix D is required during

extraction as the cover image must be multiplied by D to obtain the secret message.

The number of pixels used to hold the secret message is nearly the total number of

flippable pixels.

Gou and Wu (2007) suggested the “super-pixels” concept to improve the

embedding payload of the WPC method proposed by Fridrich et al (2005). The super-

pixels are a set of non-flippable pixels adjacent to a flippable pixel where their binary

values can be altered together without causing any visible distortion to the marked

image. Each neighbouring pixel of the k flippable pixels that matches one of pre-set

patterns can be chosen as a super-pixel. A continuous vertical or horizontal line of

zeros or ones is a simple example of the allowed patterns. The total number of pixels

to hold the secret image increases by t super-pixels. Gou and Wu have tested a

binary document image with the wet paper coding and the super-pixel techniques.

The data hiding capacity increased 6.7% by using the super-pixels approach without

any visual influence on the host binary image. Most of the selected super-pixels were

grouped in vertical or horizontal lines located on the strokes of the characters or

objects.


56

Both techniques can encode a secret message or a watermark in a binary image.

However, these techniques are sensitive and the embedded data cannot be extracted

if the marked image has been printed and scanned again.

Liu et al (2007) suggested a content-based watermarking algorithm for binary

image authentication. It divides the image into 2 parts P1 and P2 using a random

mask K. The Zernike Moments Magnitude (ZMM) is used to extract a feature vector

Z1 from the first part P1 of the divided image and Z1 is then quantized and embedded

in the other part as a watermark. The time complexity of the ZMM is very high, for

instance, the computational time of a ZMM vector for a 256x256 greyscale image is

approximately 5 minutes using MATLAB on a 1.5 GHz Pentium machine (Liu et al,

2007). In authentication, the same mask K1 is also used to split the watermarked

image to two parts P1 and P2. The feature vector Z2 is then generated from P1 using

the ZMM and quantized. On the other hand, Z1 is extracted from P2. The test image

is authentic only if Z1 and Z2 are equal; if not, it is considered as forgery.

The pixels of P1 and P2 are distributed randomly and close to each other.

Therefore, both will be altered if the watermarked image is modified. As a result, the

extracted vectors Z1 and Z2 cannot be identical and the authenticity of the image will

be rejected. However, the sensitivity of the watermarks generated by this algorithm

makes it unable to distinguish between malicious and non-malicious attacks.

Consequently, it fails to authenticate images after print and scan. Also, it cannot

determine the locations of the modified parts of images.


57

2.5.3 Other Techniques

Kwok et al. (2000) proposed watermark design patterns that help watermark

technique developers to choose an efficient technique for electronic commerce

copyright protection among available watermarking techniques depending on the

requirements of system and its data type.

Online distribution of multimedia content, broadcast services, document

verification, ownership identification, electronic publishing and advertisement, real-

time information delivery, product ordering, transaction processing, photograph

galleries, digital libraries, and digital newspapers and magazines are some examples

of electronic commerce applications (Kwok et al., 2000). The data types used in

these applications and websites could be text, images, audio, or video.

System for Copyright Protection (SysCoP) is a data hiding algorithm proposed by

Zhao and Koch (1995) for colour, greyscale, and binary images. It divides the

cover image into 8x8 contiguous or distributed blocks and encodes a single bit in

each block provided that the embedded bit does not cause any visual noise to the

host block. A contiguous block is an 8x8 square existing as a unit and taken directly

from the original image grid while the pixels in a distributed block are selected

randomly from the original image without repetition. Distributed blocks make

detecting locations of the embedded labels harder for attackers even if they compare

different labelled images.

In colour and greyscale images, the method applies a DCT transform to each

contiguous block and then each transformed block is quantized. The middle

frequencies of the quantized DCT coefficients are then modified to hold the label bits.

Afterwards, de-quantization and the Inverse Discrete Cosine Transform (DCT) are

applied on the modified blocks to construct the labelled image. The label can be


58

extracted by applying quantization and DCT transform on the 8x8 blocks of the test

image. This method is supposed to be robust against JPEG compression because it

uses the DCT transform, which is also used in this lossy compression technique, but

it may fail to prove the ownership of the labelled images after print/scan attack.

In binary images, the rate of black pixels in each 8x8 block is computed. Then, a

bit 1 of the label is inserted into blocks which have a black rate greater than a given

threshold and a bit 0 is embedded if the rate of white pixels is less than another

predetermined threshold. The data embedding capacity of this method depends on

the content of the host image. In dithered binary images, the black and white pixels

are distributed all over the image and interweaved. Therefore, dithered binary

images can carry more data than other binary images with distinctive large flat

black/white areas where labels can only be inserted at the borders between black

and white regions. Thresholds must be chosen carefully to increase the robustness of

the inserted label and to guarantee accurate extraction. This method is sensitive to

simple attacks, for instance removing a line from the label image could cause wrong

label extraction. Also it fails to prove the copyrights after D/A conversion or image

resizing attacks.

Sun et al (2001) proposed a content-based optical watermarking technique to

authenticate document images without the need for converting the test document to

digital format. In this technique, the watermark is embedded by modulating the high

frequencies of the document. Visual cryptography with n keys is used in this method

to increase the level of security. The watermarked document can be visually verified

using a photocopier provided that the printer used to produce this document has a

higher resolution than the photocopy machine. The cryptographic keys are overlaid

during verification to distinguish between forged and original documents. The


59

embedded watermark image will appear if the photocopied document is original.

Otherwise, a noisy image will be the output from the photocopier.

This method could be useful for verifying printed documents. However, it

requires high-priced equipment such as printers with high resolution. Therefore, the

cost could be inconvenient for some organizations which find this technology

unaffordable. In addition, the number of verification sites is limited as only those

who have the cryptographic key, which could be a transparent slide, can verify

documents. Also, photocopiers with higher resolution than printers used in

embedding can cause false document verification.

In Digimarc (2009), watermarking solutions to verify the integrity of document

images, detect unauthorized alternations, and discover the source of leak are

offered. No detail about the proposed technology is given in the Digmarc website.

Signum Technologies, a partner of Digimarc, designed Signum's VeriData

software to check the integrity and validity of imaged documents (Signum

Technologies, 2009). VeriData is said to be commercial low-priced software with a

high level of security to detect unauthorized modification in (Tagged Image File

Format) TIFF and JPG document images. The software is also flexible enough to be

compatible with different systems.


60

2.6 Conclusion

In this chapter, many watermarking techniques proposed in the literature to

protect copyrights or to authenticate digital images were reviewed. The vast majority

of these techniques are suitable for colour and greyscale images due to the high data

capacity of those types of digital data. Only a minority of data hiding techniques are

proposed for binary images. However, most of them have not shown enough

robustness against print/scan attack. Therefore, those techniques can be used only

to protect digitally transmitted and manipulated binary images.

The lack of techniques to protect binary images against print/scan attacks is due

to two main reasons. First, binary images have low data hiding capacity and altering

the values in a binary image grid can cause visual noise. Secondly, digital-to-

analogue conversion can distort objects in digital images.

Chapter 3: Fundamental Methodologies and Principles

61

Chapter Three: Fundamental Methodologies and Principles

3.1 Introduction A number of essential concepts and basic techniques have been in the proposed

method of this research. Therefore, it is important to give background information

about those principles and techniques. This chapter sheds light on those techniques

and discusses them in details.

3.2 Print and Scan Model The vast majority of image watermarking techniques are fragile to the print/scan

operation. According to Yu et al (2005), the three main difficulties behind the lack of

watermarking techniques proposed to survive digital/analogue conversion are:

1- Randomness: The watermarked image after a print/scan operation looks

exactly like the original watermarked copy when seen by the naked eye but in

fact there are enough differences to damage the embedded watermark. The

variation is due to the inaccuracy of the mechanisms of printers and scanners.

2- Human-dependency: The printed/scanned document could be adjusted one

way or another by the user, printer, or scanner which may eliminate the

inserted watermark. Examples of those desirable or undesirable adjusments

are contrast enhancement, scaling, gamma correction, or orientation of

papers which may occur in the tray of a printer or the flatbed glass of a

scanner.

3- Indistinguishability: The distortion of both printing and scanning processes on

the printed/scanned materials is combined and cannot be separated because

when a paper is printed, it has to be scanned to be analyzed or processed by


62

computer. Therefore, the changes occurring in both print and scan will be

merged.

A robust watermarking technique to protect hardcopy documents is still needed in

order to change the future of copyright protection and verification technology for

paper media such as books and newspapers. However, the print-and-scan issue is

still not resolved. The Print-Scan model is shown in Figure (3.1) in Yu et al (2005).

Low-pass filter and geometrical distortion are introduced to the document in both

print and scan phases. But, there is more geometrical distortion during the scan

process than in printing.

Figure 3.1 : The Print-Scan Model (Yu et al, 2005)

Print and scan operations, their structure, and influences on the original media

are discussed in detail, separately, in the next two sections.

3.2.1 The Print Process

The Structure of a laser printer is shown in Figure (3.2) in Yu et al (2005). When

a document or an image is printed, distortion is very likely to happen to the printed

media and there are different sources of distortion during the print process:

Halftone

Grayscale Conversion

Attack

Geometric Distortion

Blur

Geometric Distortion

Geometric Attack

Lowpass filter attack

Blur

Print Scan

+ + Output Original Image


63

1- In the optical system of the laser printer, the lenses are supposed to work

always in linearity. Otherwise, noise could appear on printed materials if

any non-linearity is introduced.

2- The luminance bias and movement inconsistency of the laser source can

also cause noise in the printed materials.

3- Blur of the hardcopy could be caused by dot gain which results from the

optical system and from colorant spreading.

4- Continuous greyscale to binary pattern conversion causes significant

differences between the original and the printed copy.

5- Geometrical image displacement could happen if there is non-uniform

distribution of toner. This displacement will not happen if the Organic

Photo Conductor drum and all the rollers in the Electrophotographic system

are perfectly cylindrical with a smooth surface.

Figure 3.2 : The Structure of Laser Printer (Yu et al, 2005)

Raster Image Processor

Buffer

Actuator

Laser Source

Parabolic lens

F-θ lens

Organic Photo Conductor drum

Developing Roller

Charging Roller

Transferring Roller

Fusing Roller

Pressing Roller

Laser Printer

Original Image

Hardcopy

Control System

Optical System

Electrophotographic System

Latent image


64

3.2.2 The Scan Process

The scan process also has a major impact on the details of the hardcopy. The

structure of the scanner is shown in Figure (3.3) taken from (Yu et al, 2005). The

mechanism of a scanner can be summarised as follows, the reflector sheds the light,

produced from the light source, onto the whole glass of the flatbed where the

document is placed. The luminance of the reflected light is then converted to signals

by the Charge Coupled Device (CCD) and sent to the ADC system and then to the

PC. Output digital images differ from hardcopies after converting them to a digital

form via a scanner. The variation can happen at different stages:

1- The Modulation Transfer Function (MTF) in the optical system of the

scanner causes a Gaussian low-pass filter attack which blurs the scanned

document.

2- A Charge Coupled Device sensors causes thermal noise and dark

current noise which affects the quality of the scanned image.

3- The stepped motion jitter of the carriage also causes Gaussian random

noise.

4- The scanning process also causes additive and multiplicative noise and

its degree varies from a scanner to another depending on the resolution of

the scanner.

5- Users cannot avoid changes of orientation when a document is placed

on the glass of a flatbed scanner. Most likely, all documents are subject to

rotation in some degree.


65

Figure 3.3: The Structure of Scanner (Yu et al, 2005)

3.3 Centre of Gravity

The definition of the centre of gravity (also known as centre of mass, or

centroid) is a unique point for a system (one object or more) that represents the

average position of the mass of this system (Pratap & Ruina 2009; Bourke, 1988).

The centre of mass can be computed for standard geometrical shapes such as circle,

triangle, square, rectangular, oval, pentagon, and hexagon as well as for any

asymmetrical polygons. Examples of symmetrical objects are shown in Figure (3.4-a)

and asymmetrical objects are shown in Figure (3.4-b). The centroid of each object is

marked by a dot (•). The centre of gravity of symmetrical objects is located at the

axis of symmetry in the middle of the object.

Figure 3.4: The Centre of Mass of (a) Symmetric Shapes and (b) Asymmetric Shapes

Light Source

Reflector

Charger Coupled Device sensor

ADC System

Drive System

Scanner

Hardcopy Digital image

Optical system

(a) (b)


66

In engineering, it is essential to calculate the centroid of any rotating piece of a

machine such as a train, car, aeroplane, or boat accurately to stabilize the engine

and reduce or avoid vibration to the minimum level. Otherwise, shaky revolving

objects in a machine may lead to a catastrophe such as an engine explosion.

Nowadays, Computer Aided Design (CAD) software makes it easier for engineers to

calculate the centre of mass of a machine (Pratap & Ruina, 2009). The centroid is

fixed for rigid objects and it can be calculated for any 1D, 2D objects, or higher D.

To calculate the centre of mass of a closed polygon consists of N vertices (xi,yi)

where (0 ≤ i < N) and the vertex (x0,y0) and (xn,yn) are the same point. An

example of a closed polygon is shown in Figure (3.5) in (Bourke, 1988).

Figure 3.5: Closed Polygon with 6 Vertices

The area (A) of the polygon must be computed first by equation (3.1):

The centroid point (Cx, Cy) can be calculated by using equations (3.2) and (3.3)

respectively.

…. Equation 3.1 (Bourke, 1988)


67

If an object has one or more inner holes, its centre of mass can also be

computed using the above equations, provided that the areas of the inner holes are

calculated and subtracted from the total area of the object.

According to (Bourke, 1988) these equations cannot be used to calculate the

centre of mass of self-intersecting polygons such as those shown in Figure (3.6).

Figure 3.6: Self Intersecting Polygons

This limitation therefore makes this technique inapplicable to some objects which

may exist in the system. In addition, if this method is applied on a digital image

containing many objects with a large number of vertices, the computational time to

detect the vertices and calculate the cenroid of each object in that image will be

long. Therefore, it is better to use a simpler and faster technique to compute the

centre of gravity of objects within an image.

3.4 The Method of Moments

The method of moments is a simpler and faster technique to compute the centre

of gravity of objects within an image. It is also preferable for use as an alternative

way to calculate the centroid of a binary image. Consider all the black areas in the




68

binary image I as objects (forground) and the white areas as the space

(background). If the mass of each black dot in the binary grid equals 1, the moment

of each dot (x,y) in the grid is the multiplication of the mass m and the directed

distance d from both the x-axis & y-axis as (moment = m × d). The average of the

moments of each dot is then computed to calculate the centroid (Xc, Yc) of the whole

M×N image as shown in Equations 3.4 and 3.5 below (Dai et al, 1992; Li, 1993;

Flusser, 1998; Chung & Chen, 2005):

Where T is the number of black pixels in the image I, f(x,y) is the mass of the

black point (x,y) and it always equals 1, M is the height, and N is the width of the

image I. This technique can be used to work out the location of the centroid of the

whole image or of a particular object in an image if it was processed individually as a

separate image. Its computational time is relatively short.

3.5 Quad Tree

A Quad tree, also known as Q-tree, is a data structure named by Finkel &

Bentley (1974) and defined as “the Simplest and most powerful geometric data

structure” by Har-Peled (2008). The Q-tree is preferred over other data structures

because it is easy to implement and fast for data search. In two-dimensional space,

the root region is partitioned by vertical and horizontal cuts into 4 square or

rectangular quadrants called children. The generated children are then recursively

... Equation 3.5

... Equation 3.4


69

sub-divided until certain conditions apply. Each node has either 0 or 4 children. A

node that has zero children and cannot be split any further is called a leaf. The depth

of the quad tree is the number of partitioning iterations which generated the very

last leaves (the root node has depth zero) (Har-Peled, 2008; Berg et al, 2008). The

resultant grid does not have to look like a mesh because the quadrants are not

necessarily equal in size or similar in shape. An example of Quad-tree of a 2D image

before and after partitioning is shown in Figure (3.7). The depth levels of Quad tree

partitioning are shown in Figure (3.8).

Figure 3.7: An Example of Quad Tree of an Image (a) before and (b) after partitioning

Figure 3.8 : The Depth Levels of Quad Tree Partitioning of an Image

(a) (b)


70

Quad trees can be used in a variety of applications such as (QuadTree, 2009):

1. Spatial indexing.

2. Image representation.

3. Efficient collision detection in two dimensions

4. Viewing frustum culling of terrain data

5. Storing sparse data, such as a formatting information for a spreadsheet or

for some matrix calculations

6. Solution of multidimensional fields (computational fluid dynamics,

electromagnetism).

3.6 The 2D Data Matrix Barcode

The 2D barcodes such as DataMatrix ECC200 are more desirable than 1D barcodes

not only because of their data capacity but also due to their robustness. The encoded

data can be recognised even if the 2D barcode is up to 60% corrupted because a

Reed Solomon Error Correction Code is used (Hen et al, 2005).

Applications with a large data capacity labelling requirement such as Direct Part

Marketing and Product Marketing use 2D barcodes due to their large data capacity.

The 2D barcode is constructed of 4 main components as shown in Figure (3.9):

1- Solid-Line Locator: this is an L-Shape outline that shows the proper

direction of the code and the boundaries of the data area

2- Patterned-Line Locator: two black and white lines making a corner at the

side opposite the solid-line locator lines. They specify the number of rows

and columns in the barcode as well as the boundaries of the data area.


71

3- Data Area: the binary data is repeatedly encoded in this area to guarantee

successful decoding even if some parts of this area have been corrupted.

4- Quiet Zone: is a vacant region with at least 1.5 inches breadth surrounds

the entire barcode; therefore, no data are included in this area.

Figure 3.9: The 4 Components of the 2D Data Matrix Barcode

The 2D data matrix barcode is preferred for several reasons (The IET, 2005):

1- High data capacity: can hold an abundance of encoded data (e.g. 3116 digits

or 1556 American Standard Code for Information Interchange (ASCII)

characters in a small area.

2- Flexible encoding: all kinds of data such as numbers, texts or ASCII code can

be encoded into the data matrix.

3- Adaptable: as barcodes with only 20% contrast can be correctly read, it can

be printed on any type of material.

4- Robust: Solomom Error Correction Code is used in the most widely used ECC-

200 2D barcode; thus, encoded data even in the most of damaged barcodes

can be properly recognized by barcode readers.

5- Orientation-independent: barcode readers can read encoded data in rotated

barcodes at arbitrary rotation angles.

6- Multiple-asset automatic identification: hundreds of unique codes can be

encoded and detected from a single image. Furthermore, barcode readers

have high speed in recognizing encoded data.

1- Solid-Line Locator 2- Patterned-Line Locator 3- Data Area 4- Quiet Zone


72

In this research, the 2D date matrix barcode has been chosen for the following

reasons:

1- The limited data hiding capacity in binary images as each pixel is represented

by a single bit.

2- The large amount of noise which could occur in the host binary image after

embedding. This noise may spoil the content of the original document.

3- The sensitivity of data hiding techniques to a print⁄scan operation which can

significantly change the grid contents of the binary image.

4- The visible attached barcode may deter the counterfeiters from altering the

protected document.

3.7 Type I and Type II Statistical Errors

Type I and Type II statistical errors were used in this thesis to describe the

inaccuracy of the verification system of the proposed methods in this thesis. These

types of errors show how the verifier can accept or reject a scanned document. The

statistical glossary by Easton & McColl (1997) shows that the two possible errors can

be either:

1- Type I error (Also known as Type-α or False-Positive error) when a null

hypothesis is rejected when it is actually true. In this research, false positive

errors happen when an authentic scanned document is rejected and considered

as a forgery by the verifier.

2- Type II error (also called Type-β or False-Negative error) when a null

hypothesis is not rejected when it is actually false. In the methods proposed in

this thesis, a false negative error happens when a forged document is accepted

and treated as a genuine copy by the verifier.


73

Table (3.1) in Rogers & Link (1998) shows the statistical errors for true and

false hypothesis test.

Null Hypothesis True

Null Hypothesis False

Reject Null Hypothesis Type I error Correct

Fail to Reject Null Hypothesis Correct Type II error

Table 3.1: Hypothesis Test in Statistics

The methods proposed to validate document images are discussed in detail in

the next chapter.

Chapter 4: Methodologies

74

Chapter Four: Methodology

4.1 Introduction

In this chapter, two different methods to create and verify self-validating

documents are proposed and the theory of each method is discussed in detail. The

advantages and disadvantages of each method and the similarities and differences

between these two methods are also given.

4.2 Method 1

In this method, document images need to pass through the creation and

verification stages. To create a self-validating document, the preservative data are

derived from the area of interest in the document, encoded to a barcode, and then

the barcode is attached to the document itself. In the verification stage, the

preservative data is extracted from both the document contents and the attached

barcode and a comparison between those extracted preservatives will decide

whether the scanned document has been significantly altered or not.

4.2.1 The Creation of Self-validating Documents

To generate a self-validating document, a digital document has to pass through

several stages. First, the area that counterfeiters may aim to alter, in order to

achieve illegitimate benefits, needs to be manually chosen. The selection of this area

has to be done by the creator of the document. This area may contain the signature

of a person, the stamp of an organisation, or any numbers, text, or a combination of


75

both that symbolize a name, a date, an amount of money, or any valuable

information in a document.

The selected area of the document image is then converted to a binary format if

the creator has accidently chosen an image that is not already in black and white

format. The binarization can be simply applied on a greyscale image I by setting any

pixel with a value less than a predetermined threshold (Tb) to a black pixel (set to

zero) in the binary image Ibinary. Otherwise, the pixel is converted to a white pixel in

the new binary grid. As shown in equation 4.1.

In true colour images (24-bits), the luminance of the colour image must be

calculated first using equation (4.2) to convert the RGB palettes to a grey-level

image Y (Koschan & Abidi, 2008; Pratt, 2007):

Y= (0.299×R) + (0.587×G) + (0.114×B) … equation (4.2).

To convert the values of the Y grid ranging between 0 and 255 to a binary-level

image, Equation 4.3 is used. The threshold can be set by the software designer

depending on the system requirements:

The binary text image is then scanned horizontally in order to detect the number

of lines in the image. If there is more than one line, the image is then split into

separate lines. The detection of lines is based on finding at least a white line (i.e.

… Equation (4.3). Ibinary(x,y) =

0 if 0 ≤ Y(x,y) < Tb

1 if Tb ≤ Y(x,y) ≤ 255

… Equation (4.1). Ibinary (x,y) =

0 if I(x,y)< Tb

1 if I (x,y)≥ Tb


76

without any single black pixel) between any two sets of rows containing black pixels

and separating those two sets of rows into individual images. Afterwards, each line is

divided into separate characters or objects by scanning it vertically. The principle of

an objects/characters scanner is similar to the lines separator. If there is a single

white vertical line, or more, between a character and another neighbouring

character, these two characters are split into 2 separate sub-images to be processed

individually later. Unlike the connected component analysis in Tan et al (2002) which

only divides only lines into separate parts if they are connected, this method divides

lines into objects if there is at least one white line between characters.

The centroid (Xc,Yc) and the black pixel rate (Rb) of each character/object is then

computed, the values of Xc, Yc, and Rb range between 1 and 100%. An example of

computed Xc, Yc, and Rb values for the letter ‘O’ is shown in Figure (4.1).

Figure 4 : An Example of Computed Centroid Point (Xc, Yc) & the Rate of Black Pixels of a Binary Image Containing the Letter “O”.

The value Rb has a range between 0 and 100%. If the image is totally white (i.e.

there is no object in the image), Rb=0. But if it is totally black, Rb=100. Otherwise,

Rb can be any number ranging from 1 to 99%.

As Xc & Yc represent the location of the centroid of an object, or set of objects, in

an image, the values of Xc & Yc must be in the range 1 to 100% because the centroid

point has to be inside the image itself. If the centroid is in the top right corner, then

(Xc=1% and Yc =1%). If the centroid is the last pixel in the right bottom corner, then

(Xc=100%, Yc=100%) and those are the maximum values of Xc & Yc.

The Centroid is in the middle of the object. Therefore the values of Xc= 50%, Yc =50%. The rate of black pixels Rb = 34%


77

Therefore, it requires at least 7 bits to represent each value of Xc, Yc, and Rb in

a binary stream. In this method one byte is used to represent each value.

This binary stream representing Xc, Yc, & Rb values is the preservative data of the

area of interest in the document. Each object/character requires 24 bits of data to be

represented in that stream. This data stream is then encoded into a 2D barcode. The

barcode that carries the preservative data is then attached to the document image in

a blank white area to generate a self-validating document and only documents with

barcodes can be printed and distributed. Figure (4.2) shows the process of creating

self-validating documents.

Figure 4.2 : A Block Diagram of Creating a Self-Validating Document

4.2.2 The Verification of Self-validating Documents

The verification and creation procedures have some stages in common. In

verification, the distributed hard copy document needs to be converted to the digital

format first by using any optical scanner. After that, the scanned document image

(I‘) is converted to the binary format in the same way during the creation stage

because the verifying user may scan the document as a colour or a greyscale image.

The area of interest which might be forged is then chosen by the user. Afterwards,


78

the selected part is scanned horizontally line by line in order to detect undesirable

minor noise such as dots caused during printing, transmission, or scanning.

A noise threshold Tn for the noise must be pre-determined and any rate of noise

lower than this threshold will be eliminated from the document. The value threshold

can be set by measuring the maximum possible number of lines in a document to

determine the average height of the line. Tn has to be significantly less than the

average height of lines. In the horizontal scan, if the height of a line is lower than Tn,

it will be considered as noise and consequently removed from the document. If there

is a single dot or a small number of added dots in the scanned document, possibly

caused by some dirt on the glass of the flatbed scanner, they will be erased by this

noise removal process. An example of a scanned image containing noise caused by

a scanner is shown in Figure (4.3a) and the same image after the noise removal

process is shown in Figure (4.3b)

Figure 4.3: A Sample Scanned Image (a) before & (b) after Noise Removal Process

Afterwards, the same line splitter applied in the creation stage is used to divide

the selected parts into separate lines. In addition, the vertical scanner used during

creation is also applied to divide each line into separate characters and to save each

character as an individual binary sub image. The centroid point (X‘c, Y‘c) of each sub-

image and the rate of black pixels (R‘b) are then computed.

Afterwards, the data stream previously encoded in the 2D barcode attached to

the scanned document is extracted using a 2D Data Matrix barcode reader.


79

If the size of the generated stream (S‘) from the scanned document and the size

of the other stream (S) extracted from the barcode are different, the document

image is considered as a forgery. There are two main reasons for this variation in a

stream size:

1- One or more undesirable characters has been added to the document. In this

case, the size of (S‘) is larger than the size of (S).

2- Some details have been eliminated from the document. The size of (S‘) in this

case must be smaller than the size of (S).

The malicious add/remove operation could be done by a counterfeiter after the

document was printed out and distributed and before scanning it back to the

computer for verification.

Once the difference in size between (S) and (S‘) is detected, the verifier will

reject the scanned document and there is no need to take further action in the

verification process. However, a sequential byte-by-byte comparison between the

two data streams can be applied to locate the position of the added/removed data in

the scanned document.

If (S) and (S‘) are equal in size, the extracted values of Xc, Yc, & Rb of each

character/object in the data stream stored in the barcode (S) are consecutively

compared with the values X‘c, Y‘c, & R‘b calculated earlier from the scanned document

image I‘.

Two pre-determined thresholds (Tcentroid & Tblack) are needed during the

comparison to verify the authenticity of the scanned document. Tcentroid is used to

measure the difference between centroid values while Tblack is used to compare the

rates of black pixels.


80

If (|Xc - X‘c| ≥ Tcentroid) or (|Yc - Y‘c| ≥ Tcentroid) of a particular sub image, it means

the content of this sub image in the scanned document has been tampered with and

its centroid point has been significantly shifted in a vertical, horizontal, or diagonal

way.

In addition to the centroid points comparison, the difference between each rate of

black pixels (R‘b) in S‘ & (Rb) in S needs to be computed to measure how significant

the change in the number of black dots in the sub image is. The comparison of black

rate is essential in case the centroid comparison fails to detect any changes in the

image and vice versa. The Tdiff threshold is used to decide whether the content of the

scanned sub-image has been significantly altered or not.

If |Rb - R‘b| ≥ Tdiff, it means that there are many black pixels have been added

to/removed from the scanned sub-image. As a result this part of the document will

be considered as a forgery and consequently the whole document will be rejected.

An example of a shifted centroid position and a significant change in the rate of

black pixels of a character before and after forgery is shown in Figure (4.4). The

white dot in the character represents the centroid points.

Figure 4.4 : An Example of a Character (a) before and (b) after Forgery, its Centroid, and Black Rate

(b) Forged Black Rate = 67.6% Centroid X = 49% Centroid Y = 49%

(a) Original Black Rate = 57% Centroid X = 54% Centroid Y = 49%


81

The differences between the values of those characters in the example above are

computed. The difference in black rate = 10.6%, centroid X = 5%, and centroid Y =

0%.

There must be at least one or more significant differences exceeding the

predetermined thresholds between the values of S and S‘ in order to consider the

scanned document as a forgery. Otherwise, the content of the scanned document is

considered as authentic. The verification process is shown in Figure (4.5).

Figure 4.5 : The Verification Process of a Scanned Document

The creation and verification processes of self-validating documents using

method 1 are discussed previously in this section. The next section will discuss

another method also used to create and validate document images.


82

4.3 Method 2

In this section, the second method to generate and authenticate self-validating

documents is presented. Different procedures are used in this method. A quad tree

technique is used to divide the parts to be protected recursively and to generate the

preservative data during both the creation and verification processes. This

preservative data is then embedded in a barcode and attached to the document

itself. A comparison between the data in the barcode and the data extracted from the

scanned image is required to verify the authenticity of the document. The creation of

self-validating documents will be explained in detail in section (4.3.1) while the

verification process is discussed in the subsequent section.

4.3.1 The Creation of Self-Validating Documents

The process of creating self validating documents consists of several steps. First

of all, the area of interest needs to be manually selected by the creator. Two

bookmarks are then added to the top-left and bottom-right corners of the selected

area. Those bookmarks are essential to specify the exact area of interest and will be

needed later during the verification process. The selected part may need to be

converted to binary format if it is initially in colour or grey scale format.

Initially, a scan is made to detect whether there is an object/character or not in

the chosen black and white image. If the image is totally white (i.e. it does not

contain any black pixels), it will be ignored and no further process is needed because

there is no point in protecting an empty image. Otherwise, if the image has at least

one object, it will pass through the rest of the creation procedures. The area of

interest also needs to be checked to see if it is totally black. It is unlikely to be totally

white or totally black but if this happened, the image would not be processed.


83

The centroid point (Xc, Yc) of the entire binary image is calculated using the

technique discussed earlier in (section 3.4). If the centroid location is close to the

margins of the image, no further processing of the image is required to avoid

obtaining malformed images with large height and insignificant width or vice-versa.

This could happen only when the object location is close to one of the corners or the

side edges of the image. A predetermined threshold Tmargin is required to measure

how close the centroid is to the image border. Two thresholds (Tx & Ty) are derived

from Tmargin to be compared with the values of the centroid point of Xc & Yc. The

values of these thresholds are variable depending on the dimension of the image (I)

being processed. Tx can be calculated by multiplying the width of (I) with Tmargin and

dividing the result by 100 while the value of Ty is derived from the multiplication of

Tmargin with the height of the image (I) and the total must also be divided by 100. The

calculation of Tx & Ty is shown in equation (4.4) and (4.5) respectively.

Tx = Width (I) × Tmargin ÷ 100 … Equation (4.4)

Ty = Height (I) × Tmargin ÷ 100 … Equation (4.5)

An example of a (10×10) image is shown in Figure (4.6) where Tmargin is set to 10

in (a) and 20 in (b).

Figure 4.6 : An Example of a (10×10) Image where Tmargin = 10 in (a) & Tmargin = 20 in (b)

(a) (b)


84

In Figure (4.6a), the centroid is within the accepted region (the white area)

because (Tx < Xc < Width (I)- Tx) and (Ty < Yc < Height(I)- Ty). Therefore, the image

(I) can be passed to the next procedure (the quad tree), which will be discussed later

in this section. Otherwise, the centroid point is in the grey area of the grid as in

Figure (4.6b) and no further process is applied on the image (I).

If all the conditions above are applied on the image (I), it will be divided

vertically and horizontally into 4 parts (P1, P2, P3, P4) from its centroid as a quad

tree. The quad tree partition is discussed in Chapter 3.

After that, the height and width of each divided sub-image the 4 new parts are

compared with the height and width of the original area of interest selected earlier. If

the size of either the height or the width of any part is relatively small, no further

processing is needed to this particular part. A threshold TS is used to compare the

dimensions of (P1, P2, P3, P4) with the size of the area of interest. Let P and A denote

a sub-image and the area of interest respectively. The sub-image P is undersized if

one of the two following conditions applies:

- The width of P < (The width of A × TS /100)

- The height of P < (The height of A × TS /100)

The resulting 4 parts are then processed individually in a recursive loop until one

of the conditions above does not apply for any sub-image undergoing processing.

The recursion of the document creation process is shown in Figure (4.8).

The rates of black pixels (Rb) of all divided sub-images are then computed and

represented sequentially in a data stream. Each value of the black rate is stored in

one byte. Therefore, the size of the preservative data of n sub-images equals n

bytes. This preservative data is then encoded in a 2D barcode and attached to the

image itself.


85

If the size of the preservative data is too large to be encoded in the barcode, the

values of the black rates of each sub-image can be represented in 2-bits instead of

8-bits in order to diminish the size of the preservative data to the extent that

barcode capacity will be sufficient for holding this data.

To reduce the size of the preservative data (D), two thresholds T1 and T2 are

needed to encode the 8-bit values of D into 2 bits each. The values of D are ranging

between (00000000) and (01100100) in binary when the one byte-per-value

representation is used. When the 2-bit representation is applied, each value (v) in D‘

will be equal to one of the following binary values (00, 10, or 11). Equation (4.6) is

used to convert the 8-bit values of D into 2-bits values.

The converted values of (D‘) have 3 different values and the meaning of each

value is given below:

i. White (00): means the colour of the image is mainly white because it contains

more white than black pixels. The rate of black pixels in these images must be

between 0 and T1.

ii. Uncertain (10): means the image has a critical/delicate mixture of black and

white pixels. The rate of black pixels must be in the range T1 to T2.

iii. Black (11): means the black is dominant in the image because the majority of

the pixels are black. The rate of black pixels must be between T2 and 100.

00: if 0≤ D‘<T1

10: if T1≤ D‘ ≤T2

11: if T2< D‘ ≤100

D‘ =

…. Equation (4.6)


86

Once the preservative data is converted from the 8-bit to 2-bit format, it will be

impossible to recover the previous format because this conversion works as a lossy

data compression method.

Assume D is a data stream contains n values (v1, v2, v3, v4, ...,vn) where each

value in this stream is in the range 0 and 100. The size of this data stream will equal

n×8 bits. This data size can be compressed to the size n×2 bits. Figure (4.7) is an

example that shows how the values in data stream D are compressed where T1=35

and T2=50 in this example. The size of the output data stream (D‘) is a quarter of

the size of the input stream (D).

Figure 4.7 : An Example of the Data Compression Technique Used to Convert the 8-bits/value into 2-bits/value data stream

The values (v1, v2, v3, v4, ...,vn) in D represent the rate of black pixels in the

images divided earlier. Therefore, a higher value means there are more black pixels

in this image.

The compressed binary stream D2 will be used as a data representation of the

area of interest previously chosen. This short data stream will be encoded in a 2D

barcode instead of the larger uncompressed stream. The barcode is then attached to

a blank area in the document to be used later in verification. Only documents with

0

T1= 35

T2=50

100

8-bit data stream D v1 = 39 (00100111) v2 = 12 (00001100) v3 = 83 (01010011) v4 = 55 (00110111)

.

.

. vn = 41 (00101001)

8-bit data stream D‘ v1 = 10 v2 = 00

v3 = 11

v4 = 11

.

.

.

vn = 10

Input Output

00

10

11


87

attached barcodes can be distributed. The whole creation process of self-validating

documents is shown in Figure (4.8).

Figure 4.8 : A Flowchart of the Creation Process of Self-validating Documents

Attach the barcode to the document (I)

Self-Validating Document (Ì)

Hide D in a 2D barcode

Compress D

Yes

No

Yes

Yes

No

Compute Centroid C

Yes

Is the image totally white or totally black?

Document Image (I)

No Is A in binary format?

Select area of interest A

Binarize A

No

Is C close to the

margins?

Divide the image into 4 parts (Quad Tree)

Is the image too

small? Yes

Compute the rate of black pixels of the sub-images & generate the

preservative data D

Is D too

large to be saved in a barcode?

No


88

4.3.2 The Verification of Self-validating Documents

In verification, the hardcopy self-validating document initially needs to be

scanned to the computer as a black and white image. If the image is scanned in

colour or greyscale format, it must be converted to binary format. After that, the

area of interest located between, and including, the registration marks must be

manually selected by the user by using a computer mouse.

The first process is to locate the two registration marks by scanning the top-left

and bottom-right corners of the selected sub-image. The image is then cropped to

eliminate the bookmarks and retain the precise area of interest which was selected in

the creation process.

The main reason for using registration marks is to locate the area of interest

accurately and to avoid any unintentional selection of additional pixels while selecting

the area of interest. The additional pixels would cause an increase/decrease in the

rate of black pixels of the selected part which would lead to failure in verification. An

example of an image before and after the registration mark removal is shown in

Figure (4.9).

Figure 4.9: An Example of Selected Area of Interest (a) with Registration Marks (before cropping)

(b) without Registration Marks (after cropping)

In the case where no registration marks are detected, the system will make

four scans to detect the border of the image as described below:

(a) (b)


89

1- A top-to-bottom scan from the left side of the image to search for the first

black pixel (x1, y1).

2- A right-to-left scan from the top of the image to search for the first black

pixel (x2, y2).

3- A bottom-to-top scan from the right side of the image to search for the first


4- A left-to-right scan from the bottom of the image to search for the first


Afterwards, the system will consider the point (x2, y1) as the top-left corner

point in the image and consider the point (x4, y3) as the bottom-right corner point of

the image. Then, the image is cropped from those points to form another image

which will be taken to the next stage of process. Figure (4.10) shows an example of

how Method 2 crops an image with no registration marks.

(A)

(B)

Figure 4.10: An Example of Cropping an Image With No Registration Marks Using

Method 2 (A) Before Cropping (B) After Cropping

The same procedures, previously discussed in the creation process, are then

applied on the cropped part of the image. First, the content of the image is checked

and no further processing is required if it is totally black or totally white. Otherwise,


90

the centroid point (Xc , Yc) of the image is computed and checked to detect if it was

close to the margin of the image using Tmargin. If not, the quad tree process is made

to divide the image into 4 parts as discussed in (section 4.2.1).

The size of each divided part is then compared against the size of the cropped

area of interest. If the width or the height of any of these parts is relatively small

comparing to the original document image, no more processing is needed to this

particular part. Other parts are then processed recursively as shown in Figure (4.11).

When the above mentioned recursive loop is done and there are no more sub-

images to split, the rate of black pixels of each sub-image is then calculated. The

consequent stream of black-rate values represents the preservative data D2 of the

scanned document. The size of the preservative data is then checked to determine

whether it is too large to be saved in a barcode or not. If so, the same data

compression used during the creation process in section (4.2.1) is used to diminish

the size of the preservative data.

The preservative data D previously embedded in the attached barcode is then

extracted. A comparison between this extracted data D and the newly generated

data stream D2 of the scanned document is then made in order to detect any

significant modifications in the scanned document. The values in D & D2 can be

represented by either 2 bits or 8 bits each.

In the 8-bit representation, each value in both D & D2 is represented in one byte.

Initially, the size of D is compared first to the size of D2. There are 2 different results

from the size comparison. D > D2 or D < D2 this means the scanned documents has

been significantly altered and the verification system will reject it and consider it as a

forgery. However, if D & D2 are equal in size, a byte-to-byte comparison between D &

D2 is required. A threshold Tdiff needs to be set to measure how significant the

change in the document is.


91

Assume that D contains n bytes (B1, B2, B3, B4, …, Bn) and D2 contains similar

number of bytes (B‘1, B‘2, B‘3, B‘4, …, B‘n). The comparison to detect significant

modifications in the scanned document will be as follows:

The verifier can detect the number and locations of considerably modified parts

of the scanned document by using the algorithm above.

The sensitivity of the verifier to detecting forgery depends on the value of the

threshold Tdiff. Increasing the value of Tdiff makes the verifier less sensitive to

detecting modified parts. The threshold Tdiff can be adjusted to reach the optimal

degree of sensitivity which leads to an efficient verification tool.

On the other hand, the comparison in the 2-bit per sub-image representation is

different. The values in D and D2 can be one of the following: white (00), uncertain

(10), or black (11). A comparison of a pair of bits from D with another pair of bits

from D2 is needed in order to detect which part has been significantly changed. Table

(4.1) shows all the possibilities of the comparison and how the verifier decides

whether a certain part has been altered or not.

Only if a pair of bits with the value of (white 00) is switched to (black 11) or

vice-versa, the change in this part of the document image will count as a significant

alteration and, as a result, the document will be rejected. Otherwise, this part will be

accepted as an unaltered genuine image.

for i = 1, 2, 3, …, n

if |Bi - B‘i| > Tdiff : the ith part of the scanned document has been significantly altered.

otherwise: No noticeable changes are made to the ith part of the scanned document.


92

D D2 Comparison result

White 00 White 00 Genuine

White 00 Uncertain 10 Genuine

White 00 Black 11 Forged

Uncertain 10 White 00 Genuine

Uncertain 10 Uncertain 10 Genuine

Uncertain 10 Black 11 Genuine

Black 11 White 00 Forged

Black 11 Uncertain 10 Genuine

Black 11 Black 11 Genuine

Table 4.1 : Possibilities and Results of the 2-bit Representation Comparison

The sensitivity of the verifier depends on the range of the uncertainly scale which

can be adjusted by changing the values of the T1 & T2 thresholds previously

discussed in section (4.2.1). The larger the scale is, the less sensitive the verifier is.

If the verifier is very sensitive, genuine scanned documents could be considered as a

forgery and rejected by the system.

On the other hand, the verifier will accept forged scanned documents as

authentic copies if the range of the uncertain scale is large. Therefore, the sensitivity

level of the verification system must be optimal in order to achieve a high level of

reliability in document authentication.


93

Figure 4.11: A flowchart of the verification process of self-validating documents

No

Yes

Yes

No

No Yes

Yes

Yes

Compare the size of D & D2

Compute Centroid C

Is the image totally white or totally black?

No

Is C close to the

margins?

Divide the image into 4 parts (Quad Tree)

Is the image too

small?

Crop the image and remove the bookmarks

Hardcopy Document

Scan to computer

Yes

No Is A in binary format?

Binarize A

Select area of interest A

Extract D from the attached barcode

Compress D2

Yes

Compute the rate of black pixels of the sub-images & generate the preservative data D2

Is D2 too

large to be saved in a barcode?

Are D with D2

equal in size?

Are the contents of

D & D2 equal?

No

No

The scanned document is genuine

The scanned document is forged


94

4.4 The Extended Version of Method 2

Method 2 can be slightly modified to obtain a different level of accuracy in

authentication. The only difference between the extended version and the method

discussed above is that the images are sliced either horizontally or vertically into 2

parts instead of 4 sub-images as done in the quad tree stage. The main reason for

this amendment is to avoid obtaining sub-images with very small dimensions. This

can be caused if the width of document, or the area of interest, is very much greater

than its height or vice versa, for instance, a single line document. Therefore, it is

better to divide this kind of document recursively in a vertical way. The verification

system can be developed to choose between method 2 or its extended version

depending on the height and width of the selected document.

The decision of whether the image should be divided in a vertical or horizontal

way can be made depending on the contents of the image. The distribution of black

pixels in the image needs to be calculated in order to choose in which way the image

should be divided. This can be done by scanning each line in the image vertically and

horizontally and calculating the average number of black pixels in each line. The

summation of the calculated averages is then divided by the number of

vertical/horizontal lines that contain at least one black pixel. If the value of

horizontal distribution is higher than the vertical value, the image is divided in a

vertical way. Otherwise, the image is sliced into 2 parts horizontally.

The horizontal distribution of the image I can be calculated by using equation

(4.7).

𝐷𝐷ℎ = 1𝑁𝑁𝑁𝑁𝑁𝑁

∑ 𝐴𝐴𝐴𝐴𝐴𝐴(𝑖𝑖)𝑁𝑁𝑖𝑖=1 … Equation (4.7)


95

𝐴𝐴𝐴𝐴𝐴𝐴 is the average of black pixels in each row. The average of the jth row can be

computed by equation (4.8)

𝐴𝐴𝐴𝐴𝐴𝐴(𝑗𝑗) = 1𝑀𝑀∑ 𝑐𝑐𝑀𝑀𝑖𝑖=1 … Equation (4.8)

Where C = 1 if I[i,j] =0 (i.e. a black pixel)

𝑁𝑁𝑁𝑁𝑁𝑁 is the number of the rows that contains at least one black pixel in each image

and it can be computed by equation (4.9).

𝑁𝑁𝑁𝑁𝑁𝑁 = ∑ 𝑁𝑁𝑖𝑖=1 ∑ 𝑐𝑐𝑀𝑀

𝑗𝑗=1 ... Equation (4.9)

Where C = 1 if I[i,j] = 0

The vertical distribution can be calculated by using equation (4.10).

𝐷𝐷𝐴𝐴 = 1𝑁𝑁𝑁𝑁𝑁𝑁

∑ 𝐴𝐴𝐴𝐴𝐴𝐴(𝑗𝑗)𝑀𝑀𝑗𝑗=1 … Equation (4.10)

𝐴𝐴𝐴𝐴𝐴𝐴 is the average of black pixels in each column. The average of the ith column of

the image can be computed by equation (4.11)

𝐴𝐴𝐴𝐴𝐴𝐴(𝑖𝑖) = 1𝑁𝑁∑ 𝑐𝑐𝑁𝑁𝑗𝑗=1 … Equation (4.11)

Where C = 1 if I[i,j] =0

𝑁𝑁𝑁𝑁𝑁𝑁 is the number of the columns that contains at least one black pixel in each

image and it can be computed by equation (4.12).

𝑁𝑁𝑁𝑁𝑁𝑁 = ∑ 𝑀𝑀𝑗𝑗=1 ∑ 𝑐𝑐𝑁𝑁

𝑖𝑖=1 ... Equation (4.12)

Where C = 1 if I[i,j] = 0

An example of two different images containing different characters is shown in

Figure (4.12).


96

Figure 4.12 : An Example of Two Different Sub-images and Their Vertical & Horizontal Distributions

The sub-image in Figure (4.12a) should be sliced horizontally because its vertical

distribution is higher than the horizontal distribution. The calculation of the vertical

and horizontal distribution of this sub-image is shown below:

DV = (1/8 + 1/8 + 7/8 + 1/8 + 1/8) ÷ 5

= 0.275

DH = (5/9 + 1/9 + 1/9 + 1/9 + 1/9 + 1/9 + 1/9) ÷ 7

= 0.1746

The image is divided in a horizontal way because DV > DH. The second image in

Figure (4.12b) is sliced vertically because its horizontal distribution higher than its

vertical as calculated below:

DV = (2/8 + 2/8 + 2/8 + 2/8 + 2/8 + 2/8 + 2/8 + 2/8) ÷ 8

= 0.25

DH = (4/9 + 4/9 + 4/9 + 4/9) ÷ 4

= 0.4444

0 0

0

1/8 7/8 1/8 1/8 0 0 1/8

5/9

1/9

1/9

1/9

1/9

1/9

1/9

5

7

2/8 0 2/8 2/8 2/8 2/8 2/8 2/8 2/8

0

0

4/9

4/9

4/9

4/9

0

0

8

4

(a) Vertical distribution = 0.275

Horizontal distribution = 0.1746

(b) Vertical distribution = 0.25

Horizontal distribution = 0.444


97

4.5 A comparison between Method 1 and Method 2

The similarities and differences of Method 1 and Method 2 previously discussed

in this chapter are specified in this section.

4.5.1 Similarities of Method 1 and Method 2

The two methods have the following characteristics in common:

1- Binarization: Both methods convert the document image from the true colour

format to the binary format if the scanned image was scanned as a

monochrome image.

2- The 2D Barcode is used to carry the preservative data in both methods. This

barcode is attached to the original document to generate a validating

document. In verification, the embedded data is extracted from the attached

barcode and compared with the information of the scanned image in order to

detect any alternation.

3- Centre of Gravity: this is computed in both methods but the use mode of the

centroid is different. In Method 1, after dividing the image into characters or

smaller sub-images, the centroid is computed for each sub-image and the

location of the centroid is used as preservative data. In Method 2, the

centroid is computed for the entire protected area and the image is then

divided into parts from the centroid point. Also the centroid point values are

not used as preservative data in Method 2.

4- Rate of black pixels: Both Method 1 and Method 2 use the rate of black pixels

of sub-images as preservative data to be stored in the barcode. The stored

black rates are then compared with the black rates of the scanned document

in order to detect forgery.


98

4.5.2 Differences of Method 1 and Method 2

Method 1 and Method 2 are also different in the following respects:

1- Method 1 scans a document horizontally and divides it into separate lines.

Then, it scans each line vertically to divide the lines into characters.

However, Method 2 does not use the same division procedure.

2- In Method 2, the sub images are not divided if the centroid is close to a

margin, if the size of the sub-image is relatively small compared to the

original document, or if the image is totally black or white. None of those

conditions apply in Method 1.

3- Method 2 uses The Quad Tree algorithm to chop the image into 4 parts. This

is not used in Method 1.

4- The preservative data in Method 1 is the location of the centroid and the rate

of black pixels of each divided character. The data stored in Method 2

consists of only the black rate of the sub-images.

5- Method 1 employs a noise removal technique when it scans documents

horizontally to detect lines. Method 2 does not detect any added noise.

4.6 Limitations of using Method 1

Method 1 scans the lines separately from top-to-bottom to find spaces between

characters in order to divide those lines into individual characters. Therefore, it fails

to divide characters in some texts such as:

a) Strikethrough texts: because the line drawn in the middle of the text

connects all the letters of that text as shown in Figure (4.13). The method

will take the whole strikethrough text to the next process without dividing it

into characters.


99

Figure 4.13: A Sample of a Strikethrough Text

b) Underlined texts written in special fonts such CENA, Kozuka, Microsoft Yi

Baiti, Segoe Print, ZWAdobeF and many other font styles with connected

characters: Therefore, Method1 will take the whole underlined text line as a

single sub-image and will not be able to divide the underlined text into

individual characters. Figure (4.14) shows samples of underlined text written

in some of the abovementioned fonts. However, it is unusual to use one of

those fonts to write an official document.

Figure 4.14 : A Sample of a Strikethrough Text

c) Texts written in languages that have words with connected characters such

as Arabic, Urdu, and Farsi as shown in Figure (4.15): Method 1 will process

each word individually instead of splitting those words into characters.


100

Figure 4.15: Texts Written in (a) Arabic, (b) Urdu, and (c) Farsi languages

d) Texts written in special fonts in English language also have connected

characters like those in the previously mentioned languages. Script fonts

such as Palace Script MT, Brush Script, Edwardian Script MT, French Script,

and Segoe script MT are examples of fonts with connected letters.

Consequently, each word or group of connected characters will be taken

together as a sub-image without dividing them into separate characters.

Figure (4.16) shows some samples of English script fonts with connected

characters.

Figure 4.16: Samples of Texts Written in English Script Fonts

e) Text written in italic style is not always divided into separate characters by

Method 1 if no single white line between characters is detected. Therefore, the


101

sub-images may contain a combination of characters. Figure (4.17) shows an

example of a text line written in italic style and how it is divided using the

vertical scan for a white line of Method 1.

Figure 4.17: A Sample of Text Image Written in Italic Style and How it is Divided into Sub-images Using Method 1

f) In addition to the above-mentioned types of texts, Method1 considers a group of

any text lines as a single line if those lines are written in a text box or in a table

with solid lines. It takes the whole text box or table to the next process without

splitting the contents into characters. Also, if there was a vertical line drawn on

the side of the text lines, those lines will be taken as a single line in the line

separation stage because the method is based on finding at least a single totally

white row between text lines in order to split them. Figure (4.18) shows samples

of texts in a lined table and a text neighbouring a vertical line.

Figure 4.18: A Sample of Text (a) Neighbouring a Vertical Line (b) in a Table With Solid Lines

As a result of the failure of Method 1, Method 2 is preferred for verification of

documents written with those kinds of texts because it does not process individual

characters but deals with the whole document image.

(a) (b)


102

4.7 Modifications to Method 1

Method 1 can be slightly modified to be able to deal with documents with

connected and other non-English texts and to avoid the problem of character linkage

caused by low resolution scanners. The impact of the print-scan operation on

separate characters will be discussed in Chapter 5. To improve the verifier, each line

should be divided vertically into a number of sub-images instead of separate

characters. The number of sub-images per line can be decided by equation (4.13).

No. of parts⁄line = 𝑁𝑁𝑁𝑁𝑅𝑅𝑅𝑅𝑅𝑅( 𝑊𝑊𝑖𝑖𝑅𝑅𝑊𝑊 ℎ (𝑙𝑙𝑖𝑖𝑅𝑅𝑙𝑙 )𝐻𝐻𝑙𝑙𝑖𝑖𝐻𝐻ℎ𝑊𝑊 (𝑙𝑙𝑖𝑖𝑅𝑅𝑙𝑙 )

) ........ (4.13)

Using equation (4.13) converts the method from character-based to content-

based and divides each line into sub-images larger than character images. Also, it

reduces the size of the preservative data because the number of resulting sub-

images is certainly smaller than the number of characters.

The experimental results for use of both Method 1 and Method 2, their reliability

in detecting forgery and validating scanned documents, the statistical error in each

method, the analysis of the results and a comparison between the results of both

methods are given in chapter 5.

Chapter 5: Experimental Results and Analysis

103

Chapter Five: Experimental Results and Analysis

5.1 Overview

The methods proposed in chapter 4 have been implemented by using Borland

Delphi Enterprise Version 6.0. The verification system of each method was tested to

measure the efficiency and the reliability of those methods in verifying unaltered

scanned documents and detecting forgery in modified images. In experiments, 55

test document images with four different typefaces (Verdana, Times New Roman,

Georgia, and Arial) and several font sizes have been printed out using an HP LaserJet

8150 printer. Copies of all printed documents have been forged and rescanned to the

computer. Unaltered copies have also been scanned as monochrome bitmap images

using an HP ScanJet 2200c scanner with 300dpi (Dots per Inches) resolution. The

verification systems of Method 1 and Method 2 are then applied on the scanned

documents to check whether they were maliciously modified or not. The

experimental results, the analysis, and a comparison between the efficiency of the

methods are all shown in this chapter.

5.2 The Experimental Results of Method 1

As Method 1 divides text images into lines and divides those lines afterwards into

character objects, it is essential to analyse the effect of print-scan operation on

characters. The character image analysis is discussed in the next section and the

experimental results of Method 1 are given in the subsequent section.


104

5.2.1 Character Image Analysis

Method 1 divides text images into character images. The rate of black pixels and

the centroid location of each character are then computed and used as preservative

data. In experiments, character objects in the printed document are subject to

alternations. Additional black pixels around the edges of characters may appear due

to the low-resolution scanners. The extra pixels added to the characters may link two

characters or more with each other. Also, the added noise may fill the white line gap

between two characters. Examples of scanned characters with additional noise are

shown in Figure (5.1).

Therefore, the method will consider those connected characters as one object

which causes inconsistencies during the verification process and, as a result, the

document will be rejected by the verifier.

Figure 5.1: Samples of Connected Characters Due to the Additional Noise of Print-Scan Operation

Consequently, it is important to measure the additional noise caused by scanners

and to analyze the amount of noise in each character separately. The impact of print-

scan operation on all letters (A..Z, a..z) in the English language, digits (0..9), and 9


105

other symbols has been examined. The characters have been written in 7 different

typefaces and 9 different font sizes. The typefaces used in experiments are Calibri,

Arial, Times New Roman (TNR), Verdana, Georgia, Impact, and Latha. While the

sizes of fonts chosen for the test characters were 8, 10, 12, 14, 16, 18, 20, 22, and

24. Samples of the above listed fonts are shown in Figure (5.2).

Figure 5.2: Samples of Calibri, Arial, Latha, Verdana, Times New Roman, Implact and

Georgia fonts

The main reason behind choosing these typefaces and sizes is that they are

commonly used styles and sizes in official documents. Also, it is more reasonable to

select a font size between 8 and 24 in documents like these rather than other extra

small⁄large font sizes.

Seventy one characters includes letters (A..Z), (a..z), numbers (0..9), and some

special characters have been printed and scanned back into the computer. The

influence of the digital-to-analogue operation on characters has been assessed by

making a comparison between the centroid locations and the rates of black pixels of

the original and the scanned characters. If R is the rate of black pixels in a character

image and (X,Y) is the centroid point of this image, the difference between original

and scanned character image can be computed by:

R = | R(original character) – R(scanned character)|

X = | X(original character) – X(scanned character)|

Y = | Y(original character) – Y(scanned character)|


106

Each character has been affected to different levels depending on its font size

and style. Table (5.1) shows the average difference between the original and

scanned characters written in the 7 different font styles.

Font Style

Character

Calibri Arial Times

Verdana Georgia Impact Latha

X Y R X Y R X Y R X Y R X Y R X Y R X Y R Avr.

A 1.6 0.9 5.3 0.5 0.6 2.8 0.9 2.1 4.6 0.5 0.2 3.3 0.5 1.2 4 1.1 0.3 4.1 1.2 0.7 4 1.92

B 1.3 0.7 6 1.6 0.5 2.8 2.4 0.2 2.2 1.5 0.6 1.5 1.4 0.7 2.1 0.6 0.5 2.7 1.5 0.6 3.4 1.66

C 1.1 0.6 5 0.4 0.6 4.1 0.4 0.5 3.2 0.6 0.9 3.6 0.8 1 4 0.9 1 3.4 0.9 0.9 4.7 1.83

D 1.3 0.3 5.1 1.9 0.6 2.2 1.9 0.8 2.2 1.4 0.8 2.5 1.2 0.5 1.8 0.3 0.1 2.4 1.4 0.9 3.2 1.56

E 1.8 1.8 5.8 1 1 1.5 0.6 1.2 0.6 0.9 0.4 1.3 0.3 0.3 1.2 1.1 0.5 1 1.2 0.9 2.1 1.26

F 1.6 1 6.1 1 0.6 1 0.6 0.4 1 1.1 0.7 0.7 1.2 0.9 1 1.1 0.4 1.5 0.6 0.4 1.9 1.19

G 1 1.3 6.6 0.6 0.9 3.7 2 1 2.7 1.2 0.8 3.7 1.3 0.7 3.7 0.9 0.8 2.4 1.4 0.9 5.3 2.04

H 0.9 1 6.2 1 0.5 1 1 1.3 1.2 1 0.7 0.9 0.8 0.5 0.8 0.7 0.7 1.9 1 0.5 1.3 1.19

I 6.7 0.7 17 3.9 1.1 5.4 1.8 0.9 1.4 2.1 0.9 1.2 1.8 1.3 1.9 2.4 0.4 5 3.5 0.9 8 3.25

J 2.2 1.2 6.1 1.9 1.1 2.4 1.8 2.1 3 1.9 1.3 1.8 1.4 1.4 3 1.2 0.4 1.8 2.4 2.5 2 2.04

K 1.1 0.8 6.2 1.5 0.6 3.3 2 0.9 3.5 1.3 0.8 3.1 2 1.2 4.2 0.7 0.3 2.5 1.7 0.6 4.4 2.03

L 2.1 1.8 4.4 1.5 0.6 0.7 0.7 0.7 1.2 1 0.4 1.3 1 1.5 1.5 2 0.7 1.7 1.4 0.7 0.7 1.31

M 1 0.6 5.8 0.2 0.8 3.9 0.3 1 1.9 0.7 1.3 2 0.3 0.7 2.8 0.7 0.6 1.4 1 0.7 5.6 1.59

N 2.8 0.2 6.9 1.2 0.6 2 1 1.3 0.9 1.2 0.5 2.1 0.8 0.8 1.8 0.9 0.1 2.4 1.2 0.4 4 1.58

O 0.7 0.9 5.2 0.9 0.8 4.3 0.6 0.7 4.4 0.6 0.7 4.5 0.9 0.7 4.5 1 0.4 2.7 0.5 0.6 6 1.98

P 1.1 1 6.8 1.4 1.1 2.4 1.2 1.3 3.3 1.2 1.3 2.3 1.5 0.6 2.4 0.6 0.9 1.9 1.2 0.8 2.7 1.76

Q 0.4 1.1 3.4 0.9 0.8 3.3 0.2 0.2 4.2 0.4 0.4 3.9 0.4 0.7 3.9 1.1 0.4 3 0.7 1.4 5.2 1.71

R 1.3 0.7 7.2 2.4 0.8 2.7 2.7 1.1 3.5 1.4 0.3 3.2 1.9 0.7 3.1 0.5 1.1 2.4 1.2 0.7 2.1 1.96

S 1 0.7 6.7 0.6 0.4 5.4 1.4 1 5.5 0.7 0.7 5.5 0.8 0.7 6.2 1.3 0.2 4.2 1.1 0.7 6.9 2.47

T 1.1 1.8 3.4 0.7 1.5 0.7 1.1 1.4 0.4 0.3 1.2 1.2 0.5 1 0.7 0.4 0.4 1.4 0.5 1.2 1.3 1.06

U 2.5 1.1 4.5 0.5 1.7 1.5 1.1 1.4 1.7 1.4 1.2 2 0.5 1.8 1.2 0.9 0.1 1.3 1.3 1.3 2.9 1.52

V 0.8 1 5.8 0.8 0.7 4.9 1.2 0.9 3.9 0.7 0.9 4.1 1.1 0.9 4.1 0.5 0.7 3.5 0.3 0.7 4.2 1.99

W 1 1 5.8 0.9 0.9 5.1 0.4 0.7 4.5 0.7 0.4 5.3 0.5 0.1 5 0.5 0.4 3.8 0.7 0.7 7 2.16

X 1.3 0.7 5.6 0.7 0.9 5.9 0.7 0.7 4.4 0.4 0.5 3.8 0.4 1.1 4.3 0.7 0.9 3.8 0.5 0.7 6.4 2.11

Y 0.9 2 4.2 0.5 2.7 1.8 0.4 2.9 1.9 0.8 2.2 1.5 0.4 1 2.9 1.2 1.1 2.4 1.2 2.9 2.4 1.78

Z 1 1.5 5.6 0.8 1.1 2.8 1 1.7 3.6 0.6 0.8 2.7 1.2 1.2 4.3 1.1 0.4 4.2 0.7 0.6 2.4 1.87

a 1.8 1 7.1 2.8 0.9 4.9 2.4 0.8 5.2 1.6 1.2 5.4 2.1 1.2 6.1 1.2 0.7 3.5 1.9 1.2 7.2 2.87

b 1.9 1 6.8 1.3 0.6 4.1 1.4 0.4 3.6 1.9 0.3 3.4 1 0.7 2.2 0.5 0.4 2.3 1.3 0.6 4.1 1.90

c 1 1.1 6.4 0.9 1 5.6 0.8 1.2 6.7 0.8 0.8 5.2 1 0.9 5.7 0.6 0.8 4.1 1.2 0.8 6.8 2.54

d 1.4 0.7 7 2.2 0.7 4.1 3.1 0.7 3.2 1.6 0.4 4 2.7 0.4 3.1 1 0.3 3 2.9 0.2 3.9 2.22

e 1.7 1.1 8.1 1 0.3 5.8 0.9 0.6 5.4 1.2 0.9 4.9 1 0.9 5 1.2 0.7 3.9 0.7 0.3 7.1 2.51

f 1.4 1.1 6.9 2.6 1.4 2 0.8 3.4 2 0.6 1.8 1.3 0.8 1.1 2.3 2.2 0.9 1.7 1.1 1.6 2.4 1.88

g 1.2 0.7 9.2 2.1 0.6 5.4 0.8 0.3 5.2 1.3 0.6 5 0.8 0.8 5.9 1.6 0.3 2.4 1.9 0.3 5.9 2.49

h 2.2 1 4.7 1.6 1.2 2.2 2.7 1.4 2.7 0.7 0.6 1.4 1.2 1.1 2.2 0.7 1.3 1.8 1.4 1.1 2.2 1.69

i 7.1 1.8 14 3.3 0.2 5.9 5.2 3.2 8 3.4 0.2 6.7 2 2.3 4.6 2.1 0.2 2.3 5 0.8 7.1 4.07


107

Font Style

Character

Calibri Arial Times

Verdana Georgia Impact Latha

X Y R X Y R X Y R X Y R X Y R X Y R X Y R Avr.

j 3.9 0.7 6.1 2.3 1.6 3 1.7 0.4 3.8 1.9 1.1 1.9 0.9 1 2.8 2.3 0.2 3.8 2.8 1.6 3.7 2.26

k 1.3 0.6 5.1 1.7 0.2 3.4 1.8 0.9 3.2 1.6 0.2 2.6 1.3 0.8 3.9 0.7 0.1 3.2 1.5 0.7 5.4 1.91

l 5.1 0.4 15 4.9 0.8 4 6.7 1.6 9 3.3 0.8 6 2.8 0.4 3 1.6 0.6 4 4.2 0.6 4 3.75

m 1.9 2.2 5.7 0.6 2.3 1.4 1.3 3.1 3.6 0.9 2.2 2 1.2 2.2 3.4 0.3 1.1 1.4 0.7 2.1 4.9 2.12

n 2.1 1.6 6.9 1.6 2.4 2.3 2.2 3.4 5.3 1.2 1.7 2.2 1.4 1.9 3.8 0.8 1 2.4 1.4 2.8 2.4 2.42

o 1.2 1.3 6.4 0.9 0.6 5.1 0.9 1 6.6 0.4 0.7 5.2 0.7 0.6 6.7 1 0.7 4.3 1 0.8 7.8 2.57

p 1.9 1.1 7.4 1.2 1.1 3.8 0.7 1.6 3.7 1.6 1.8 3.3 1.4 0.8 2.8 0.6 0.7 1.2 1.3 1.3 4 2.06

q 1.6 1.3 6.3 2.7 1.2 2.6 2.7 1.9 3.7 2.4 1.1 3.1 1.9 1.2 3.3 1.1 0.8 2.2 2.3 1.3 5.2 2.38

r 1.7 2.3 5.9 0.6 2 2.2 0.9 2.2 2.2 0.8 1.7 2 0.6 2.4 2.6 0.3 0.9 2.3 0.9 2.3 3.4 1.91

s 1.3 1.3 8.7 0.6 0.7 7.1 1 1 7.4 0.2 0.4 6.6 0.7 0.6 7.3 1.6 0.4 5.9 0.7 0.3 8 2.94

t 1.8 1.2 6 2.1 1.2 1.8 2.3 0.9 1.7 0.9 0.9 1.8 1.7 1.1 1.8 1.1 0.4 2.8 1.2 0.4 2 1.67

u 1.7 1.2 5 1.3 1 1.4 1.2 1.3 1.6 1.3 1 1.8 1.6 0.6 2.2 1.1 0.7 2 1.7 1.4 2.9 1.62

v 1.1 0.7 8 1 1.3 5.7 1 0.7 4.4 1.1 1.2 5.6 0.8 0.8 5 0.8 1 4 1.1 0.9 5.8 2.48

w 1.3 1 6.4 0.6 1.1 4.7 0.7 0.8 4.9 0.8 0.8 6.3 0.9 0.6 5.9 0.9 0.7 3.7 1 0.6 7.3 2.43

x 1.4 0.6 7.4 1.4 1.3 5.9 1.3 1.6 6.7 0.9 0.6 4 0.9 1.2 5.6 1.1 0.7 4.4 1.1 1 8.3 2.73

y 1 1.1 4.8 0.8 0.4 4.6 1.3 1.4 4 0.8 0.7 3.6 0.6 1.1 4.7 1.8 0.8 3.8 1.4 1.3 5.9 2.19

z 1.8 2.1 7.8 0.9 0.9 2.8 1.9 1.8 4.7 0.7 1.4 3.8 1.2 1.3 5.1 0.2 0.2 3 0.9 1.7 3.9 2.29

0 1 1 3.7 0.6 0.9 2 0.7 0.6 1.2 0.7 0.8 0.3 0.9 1 1.2 1 0.6 1.7 1 0.7 4.6 1.25

1 25 9.1 4.6 1.3 1.1 1.7 9.4 2 8.5 1.1 1.4 0.8 3.3 1 2.7 1.4 0.6 1.3 1.3 0.1 3.2 3.85

2 2.2 1.4 2.8 1.2 1.3 1 1.6 1.9 1.4 1 1.3 1.1 0.4 1.6 1.1 1.1 0.9 1.7 1.1 2.4 2.4 1.47

3 1.8 0.8 2.9 1 0.6 1 1.2 1 0.9 0.4 0.6 0.8 0.9 0.6 1.1 1.1 0.6 2.6 1.4 0.9 2.7 1.19

4 2.3 2.4 1.9 1 1 1 1 1.4 1.7 1 0.8 0.9 1.1 0.9 1.4 1.4 0.6 1.6 1.6 0.8 1.2 1.23

5 1.3 1.1 2.9 1.6 0.6 1.3 1.1 1 0.9 0.9 0.6 1 1.2 0.8 0.7 1.2 0.1 0.8 2.1 1.1 3.6 1.23

6 2.9 2.2 2.4 0.9 0.8 2.4 0.9 0.7 1.9 1 0.6 0.7 0.8 0.8 0.9 1.7 0.4 1.4 1.9 0.2 4.4 1.42

7 3.8 2.4 2.8 1 0.7 0.3 0.9 0.8 1.2 0.6 0.4 0.6 0.9 0.6 0.7 0.9 0.4 2.7 5.6 1.3 2.3 1.47

8 1.3 0.6 3.9 0.9 0.7 2.8 0.7 0.7 2.4 0.4 0.9 1.2 0.9 0.7 1 1.1 0.8 1.2 0.8 0.7 3.6 1.30

9 1.1 3.4 2.4 1.1 0.7 2.1 1 0.3 1.1 0.6 0.6 1.1 1 1.1 2 1.1 0.4 1.8 0.8 1.3 4.1 1.39

$ 1 0.6 5 1 0.9 5.1 1.1 0.2 4.2 0.9 0.6 3.1 0.8 0.6 3.3 1 0.3 3.4 1 0.6 6.1 1.94

£ 1 2.3 5.1 0.8 0.4 4 0.8 0.6 4.8 0.7 2.2 3.1 1 2.6 2.1 1 1 3.3 0.7 0.6 5.4 2.07

; 3.4 1.8 5.1 3.8 2.3 5.2 3.1 2.6 6.1 2.4 2.8 1.4 1.3 1.1 6.3 2 0.3 4 1.7 1.8 5.6 3.05

. 3.3 5.8 12 5.9 5.7 9.2 6.6 6.4 25 3.6 2.3 4.2 2.6 2.1 16 2.9 2.4 2.6 3.2 5.9 8.1 6.47

, 3.6 1.4 11 5.9 3.1 9.3 2.9 1.9 12 3.2 0.7 6.6 2.4 1.4 8.3 1.6 1.2 7.4 3.1 1.4 7.9 4.59

+ 0.9 1.6 3.3 0.9 1 1.3 0.7 0.8 1.1 0.6 0.9 0.8 0.2 0.6 1.3 0.8 1.1 1.2 0.6 0.8 1.7 1.06

- 2.3 6.4 19 1.3 4.9 5 1.1 4.1 2.9 1 4.7 6.1 1.1 6.6 5.3 1.3 3.1 3.1 1.7 3.9 6.6 4.36

* 1.6 1.1 9.9 0.8 1.3 8 0.9 0.6 12 1.2 1.1 5.2 1.1 1.1 12 2 1.7 6.9 1.8 1.4 13 4.03

% 1 1.4 7.1 0.3 0.4 5.7 0.8 0.6 5.2 0.4 0.8 5.4 0.6 0.6 6.3 0.8 0.4 4.7 0.6 0.7 5.9 2.37

Average 2.1 1.5 6.4 1.5 1.1 3.5 1.6 1.3 4 1.1 1 3 1.1 1.1 3.6 1.1 0.7 2.8 1.5 1.1 4.6

Table 5.1 : The Influence of Print-Scan Operation on the Centroids and the Rates of Black Pixels of 71 Characters Written in 7 Different Font Styles


108

The average effect on each character is shown in the right hand column of the

table while the average effect on font style is given at the bottom of the table. The

table above shows that the average influence on characters differs from font style to

another. The values X and Y represent the rates of shifting in characters’ centroids

occurring in the x-axis and y-axis coordinates respectively. The value R represents

the percentage difference in the black rates of each character. Table (5.2) shows the

average influence of the print-scan operation on all characters centroids and rates of

black pixels written in font styles.

Font Style Impact Verdana Georgia Arial TNR Latha Calibri

Average effect 1.53 1.7 1.93 2.03 2.3 2.4 3.33

Table 5.2 : The Average Influence of Print-Scan Operation on Characters Written in 7 Different Font Styles Sorted from the Least to The Most Affected Fonts

The characters written in Calibri font are affected more than characters written

in the other fonts while those in Impact font are the least influenced characters.

Noticeably, the print-scan operation has more influence on characters written in

thinner fonts than those with thick strokes. The difference rates of each value (X, Y,

and R) and the average difference of the three values are shown in Figure (5.3).

Figure 5.3: The Average Influence of Print-Scan Operation on the Centroids and Rates of Black Pixels of Characters Written in 7 Different Font Styles

0

1

2

3

4

5

6

7

X

Y

R

Avr.


109

The rates of black pixels or the centroid positions of character images with small

width or small height can change more significantly than those with wide body.

Characters with the maximum and the minimum alternations in their X, Y, and R and

the values of this modification are shown in Table (5.3).

X Y R

Maximum alteration (1) 6.11% (-) 4.81% (.) 11%

Minimum alternation (Q) 0.59% (d) 0.49% (T) 1.3% Table 5.3 : The Maximum and Minimum Change in the X, Y, and R Values and their Most and

Least Influenced Characters

If the maximum alternation values are used as thresholds to compare characters

during the verification process, the scanned documents will be verified. However,

other modified characters in forged documents will also be verified if their alteration

values are less than the maximum alteration values listed in Table (5.3). Therefore,

the verification system will fail to detect a forgery which leads to a Type-I error (i.e.

False-positive). The average influence on each of the 71 characters has been

computed. Table (5.4) shows the 10 most affected characters and the average

alteration values.

Character . , - i * 1 l I ; s Average alternation 6.47 4.59 4.36 4.07 4.03 3.85 3.75 3.25 3.05 2.94

Table 5.4 : The 10 Most Affected Characters by Print-Scan Operation and the Average Difference in Their Centroid Positions and the Rate of Black Pixels

The reason why those characters are affected more than other characters is that

if the centroid point of a character with diminutive width is compared to its height,

for instance the characters (l, I, i), or a character with diminutive height compared to

its width such as (-) have been shifted to the neighbouring pixel in the coordinate of

the smaller side of the character, it causes a significant change in the location of this


110

character when compared with its original location. The value X of the letter (l, I, i,

1) is considerably altered while the value Y is slightly affected because the width in

those characters is small. In other characters such as (-), the value Y is altered more

than the value X because of its small height. The shift in the centroid location can be

caused by one or more extra pixels being added to the side of the character.

Also, the rate of black pixels in characters with diagonal strokes and edges such

as (*) or characters with curves such as (s) can be significantly influenced by

scanners. These types of characters can gain more additional black pixels than other

characters with straight vertical and horizontal lines such as (F, E, T, and H). The 10

least influenced characters and the average difference are listed in Table (5.5).

Character 8 E 0 4 5 F H 3 T + Frequency % 1.3 1.26 1.25 1.23 1.23 1.19 1.19 1.19 1.06 1.06

Table 5.5 : The 10 Least Affected Characters by Print-Scan Operation and the Average Difference in their Centroid Positions and the Rate of Black Pixels

However, the use of high-resolution scanners can reduce the potential noise in

the scanned documents.

The significant variation between original and scanned characters increases the

probability of a false-positive error during the verification process if the threshold

used for comparison has a low value. On the other hand, if a high-value threshold is

used to compare characters, modified characters will be considered as genuine and

the forged document will be accepted by the verifier. As a result, the probability of a

false-negative error will increase. The average difference between the examined 71

characters in Table (5.1) equals 2.175. If this value was used as a threshold to

compare original and scanned characters, the 10 most influenced characters will be

rejected by the verifier.


111

However, it is important to know how frequently used those characters are in

documents in order to measure the rate of error. Table (5.6) shows the frequency of

the 10 most influenced characters per 100 characters in a sample English text book

containing 9,867,698 characters.

Character . , - i * 1 l I ; S Frequency % 1.29 1.26 0.22 5.76 0.001 0.005 3.84 0.31 0.106 5.38

Table 5.6 : The Frequency Percentage of the 10 Most Affected Characters by Print-Scan Operation

The letters (i, s, and l) are the most common characters among the 10 most

influenced characters, therefore, they have more impact on the false-positive error

rates than other characters when they are altered by scanner.

It is crucial to identify what characters are of interest to counterfeiters and how

often the verification system fails to detect forgery in each character. In particular

we need to estimate the actual false-negative error but unfortunately there is no

practical estimations for the counterfeiters’ interest.

Each character is affected in different levels by digital-to-analogue conversion

when it is written at different font sizes even if the same font style is used. The

impact of the print-scan operation in the 71 characters written in 9 different font

sizes was measured by comparing the rate of black pixels (R) and the centroid (X, Y)

of the original and the scanned character images. The difference rates between the

71 tested characters are shown in Table (5.7).


112

Size

Char

24 22 20 18 16 14 12 10 8

X Y R X Y R X Y R X Y R X Y R X Y R X Y R X Y R X Y R

A 0.7 0.3 2 0.6 0.6 2.8 0.6 0.4 2.6 0.6 0.8 3.1 1.1 0.8 3.2 0.6 0.8 4.6 0.8 0.7 4.8 1.1 1.7 5.7 2.3 1.5 7.4

B 1 0.3 2 1.1 0.1 3.1 0.7 0.7 2.3 0.7 0.4 2.3 0.8 0.4 2.7 1.2 0.4 3.1 2.1 1 3 2.2 1.1 3.1 3.6 0.8 5.1

C 0.6 0.4 2.4 0.4 0.7 3 0.8 0.4 3 0.7 0.6 3 0.4 0.8 3.2 0.7 0.7 4 0.6 1 4.2 1.5 1 5.4 0.8 1.6 6.8

D 0.6 0.6 2 0.7 0.6 3.3 0.6 0.6 2.1 1.1 0.1 2.8 1.4 0.4 1.8 1.3 0.7 2.6 1.8 0.6 2.1 2 0.6 3.3 2.7 1.1 5

E 0.3 0.8 2 1 0.7 2.3 1.1 0.7 1.4 1.1 0.7 1.2 0.8 0.8 1.4 0.6 0.8 1.1 0.7 1 2.4 1.8 1.1 3.3 1.6 1.1 2.4

F 1.1 0.3 0.9 0.8 0.1 2.3 1.1 0.8 1.1 0.7 0.4 1.8 0.7 0.4 1.6 1.2 0.6 1.4 1.2 1 2.3 1.4 0.7 2.3 1 1.5 3.6

G 0.4 0.3 2.7 1 1.1 2.8 0.8 0.6 3.1 1.3 0.7 3 1.1 0.8 3.8 1.4 0.7 4.1 1.6 1.4 5 1.4 0.8 5.4 1.7 1.6 6.2

H 0.6 0.6 1.3 0.7 0.7 2.6 0.3 0.7 0.8 0.6 0.6 2.1 1 0.7 1.6 0.7 0.7 2 1.1 0.7 1.7 2 1.1 1.7 1.1 1 3.3

I 1.4 0.3 3.4 2.8 0.7 5.4 1.5 0.8 3.3 3 0.8 3.7 2.4 0.3 5.1 3.4 0.8 5.9 4.6 1 5.6 4.6 1.3 8.7 4.7 2 11

J 1.1 1 2.1 0.6 1 1.4 1.4 0.9 1.7 1.4 1.7 2.6 1.7 1.1 2.6 2.1 1.1 3.4 2.7 1.5 4.1 2.4 2.3 3.7 3 2.4 4.1

K 0.3 0.3 1.7 1.1 0.4 3.1 1.1 0.4 2.8 1.3 0.4 3 1 0.4 2.9 1.8 0.8 4.6 2.7 0.7 4.3 1.5 1.3 6 2.3 1.7 6.9

L 1.1 0.6 1 1.3 0.6 1.4 1 0.6 1 1.3 0.7 1.4 1.8 0.6 1.1 0.7 0.3 0.7 1.3 1.1 1.4 1.7 1.4 3.1 2.3 2.3 3.6

M 0.7 0.6 1.7 0.7 0.3 2.8 0.4 0.6 1.6 0.4 0.7 3.3 0.5 0.8 2.4 0.3 0.6 3.2 1 1 5 0.4 1.4 3.8 1 1.1 6

N 0.7 0.1 1.6 2 0.6 2 1.4 0.7 2.4 0.7 0.3 2.3 0.7 0.4 2.4 1.1 0.4 2.7 1 0.8 2.1 1.8 0.8 5.1 2.1 0.8 5.1

O 0.4 0.3 2.6 0.6 0.7 3.6 0.8 0.7 3.3 0.7 0.6 4.1 1 1.1 4.4 0.7 0.4 5 0.6 0.7 5.4 0.7 0.7 6.4 1 0.8 6

P 0.7 0.4 1.6 0.7 1 2.6 0.7 0.8 2.3 1.4 1 2.3 1.3 1.1 3.3 1.1 0.6 3.8 1.8 1.4 2.8 1.3 1.1 4.6 1.7 1.6 4.8

Q 0.4 0.4 2.1 0.1 0.3 3 0.6 0.7 3.3 0.6 0.3 3.7 0.6 0.6 4 0.7 0.6 3.6 1 1.1 4 0.7 1.1 5.1 0.7 1.4 5.9

R 1 0.7 1.7 1 0.6 2.7 1.2 0.6 2.8 1.4 0.1 3.1 1 0.7 3.6 1.6 0.4 3.1 2.4 1.3 3.4 2.1 0.7 5.3 3 1.7 5.6

S 0.6 0.4 3.6 1.1 0.2 4.1 0.7 0.4 4.4 0.8 0.4 4.6 0.7 0.7 5.8 1 0.6 6 1 0.7 6.5 1.1 1.1 7.2 1.8 0.8 9.6

T 0.8 0.8 1.3 0.4 1.1 0.3 0.4 1 0.7 0.7 1.6 0.9 0.6 1 0.7 0.4 1.6 1.9 0.6 1.6 2.3 1.1 1.4 2.3 1 1.1 1.7

U 0.7 0.6 1.7 0.6 0.7 2.3 1.1 0.6 1.7 1.1 1.1 2.6 1 1 2.3 0.7 1.6 2.3 1.4 1.4 2 1.1 2.4 1.4 3 1.7 3.3

V 0.7 0.4 2.4 0.4 0.6 2.8 0.7 0.7 3.6 0.7 1 3.7 0.8 0.7 4 0 1 4.6 1.3 1 5.6 1.4 1 5.7 0.8 0.8 6.9

W 0.3 0.6 2.7 0.6 0.6 4.1 0.3 0.4 4.3 0.9 0.7 3.6 0.6 0.6 4.4 0.7 0.6 5.1 0.9 0.4 5.7 1.1 0.9 7.7 0.9 0.7 9.3

X 0.8 0.6 3.3 0.3 0.6 3.1 0.7 0.7 4.1 0.4 0.7 3.9 0.3 0.6 4.1 0.9 0.4 4.4 0.4 1 5 1.3 0.9 6.4 1.3 1.6 9.4

Y 0.7 0.9 2.3 0.9 1.2 1.7 0.7 1.6 1.6 0.4 1.8 1.9 0.9 2.1 1.4 1 2.1 1.6 0.6 2.6 3.4 0.4 3.1 3.9 1.6 3.6 4.4

Z 0.3 0.7 3 0.7 1.3 3.1 0.6 0.7 2.3 0.6 0.6 3.1 0.9 0.9 3.1 1.1 1 4 1.1 1.4 3.7 2 1.9 5.3 0.9 1 5.1

a 0.9 0.4 4 1.4 0.9 4.1 1 0.9 5 1.6 1 4.7 2.1 0.9 5.6 1.9 1 6.3 3 1.2 6.3 2.4 1 7.7 3.4 1.7 7.1

b 1.1 0.4 2.7 0.7 0.4 2 1.4 0.1 2.7 0.9 0.6 2.3 1.4 0.4 4.3 1 0.7 3.7 1.9 1 5 1.9 0.6 5 1.9 0.9 6.4

c 0.4 0.4 3.4 0.4 0.7 2.9 0.6 0.6 4.6 0.4 1 5.9 0.7 1.2 5.6 1.1 1.2 6.4 2 0.7 6.7 1.1 1.3 7.7 1.1 1.1 7.9

d 1.3 0.4 3 1.9 0.7 2.3 1.7 0 3.4 1.7 0.4 3.9 1.9 0.6 3.9 1.7 0.6 5.1 2.9 0.4 4.6 3.1 0.7 5.1 3.1 0.6 5.1

e 0.7 0.7 4.1 0.6 0.4 5 0.9 0.1 4.1 0.9 0.7 4.4 1 0.6 5 1.2 0.7 6.4 1.7 1.1 7.4 1.6 1 6.4 1.3 0.7 8.7

f 0.4 1 2.9 1.1 1.7 1.4 0.6 1.6 2.1 0.9 1.7 2.7 0.7 1.4 2.7 1.3 1.1 2.6 2.1 1.4 3.7 2 2.3 2.6 3 2.3 3.2

g 0.7 0.4 3.7 1.4 0.4 4.4 1.3 0.7 3.7 1 0.4 4.6 1.1 0.4 5.3 1.4 0.1 6.1 1.9 0.4 6.4 1.7 0.6 7 1.9 1 9

h 1.1 0.6 2.1 1.3 0.7 1.1 1 0.9 1.6 1.1 1 1.6 1.8 1.2 3.1 1.4 1.1 3.4 2.1 1.1 2.1 2 1.2 4.4 1.7 2 2.6

i 4 0.7 6 2.3 1.3 4.6 3.4 1.3 6.9 2.3 1.6 6.1 4.3 1 8.1 4.7 1.4 6.4 3.7 1.1 11 5.6 1.1 8.1 6 1.7 4.7

j 1.7 0.6 3.1 1.9 0.7 3.1 1.7 0.9 3.3 1.4 1.1 3.6 2.7 0.4 3.4 2.6 0.8 3.9 2.7 0.8 2.7 3.1 0.8 4.1 2.4 2.1 4.9

k 0.9 0.4 2.7 0.9 0.1 1.9 1.1 0.3 3 1.1 0.6 2.6 1.7 0.9 4.1 1.7 0.6 5.7 1.4 0.3 4.4 2.5 0.6 6.1 1.2 0.7 4

l 4.1 0.6 6.7 3.1 0.6 4.7 3.6 0.6 4.9 3.3 0.9 6.3 4.4 1.1 6.3 4.4 0.9 11 4 0.3 4.2 3.4 0.7 8.1 6.2 1 4.5

m 1 1.4 2.7 0.6 1.7 2.1 0.1 2.1 1.7 0.7 1.9 2.9 1.1 1.6 2.7 0.9 2.6 2.6 2 2.3 3.4 1.2 2.4 5.3 1.1 3.7 5.4

n 1.1 1.4 2.9 1.3 1.9 2.6 1.6 1.7 3.4 1.1 2 3.9 1.4 1.9 4.1 2.3 1.7 4.7 1.1 3.3 3.6 2.1 2.3 4.3 1.7 2.9 3.2


113

Size

Char

24 22 20 18 16 14 12 10 8

X Y R X Y R X Y R X Y R X Y R X Y R X Y R X Y R X Y R

o 0.4 0.4 4 0.7 0.6 5 1 0.4 4.4 0.6 0.1 5.6 0.7 1 5.6 0.9 1.1 6.6 0.9 1 7 1.3 1.3 7.6 1.4 1.1 8.4

p 0.7 0.9 3.6 0.7 0.6 3 1.2 1.3 3.9 0.7 1.3 2.1 0.9 0.9 3.6 1 0.9 3.9 1.7 1.1 4.1 2.3 1.9 4.7 1.9 2 4.9

q 1.6 0.9 3.1 1.6 1 2 1.9 0.9 2.6 1.7 1 3.6 2 1.6 3.6 2 1 4.3 3.1 1.3 3.9 2.4 1.9 5 2.6 2 6

r 0.4 1 3 0.6 1.6 2.3 0.7 1.6 2.1 0.4 1.7 2.4 0.3 2 3.1 1.1 2.1 4 1.4 2.9 2.3 1 2.1 3.9 1.2 2.9 3.4

s 0.4 0.3 4.4 1.1 0.1 5.1 1 0.4 4.7 0.6 0.6 6.1 0.4 0.6 7 0.6 0.9 8.4 0.9 1.1 8.7 1 0.9 9.2 1.7 1.3 11

t 1.1 0.6 1.9 1 0.7 0.9 1.4 0.9 1.6 1.6 0.9 1.7 2 0.7 2.6 1 0.7 2.6 1.7 0.6 3.4 1.3 1.7 3.1 3.1 1.3 5.1

u 0.9 0.7 1.7 1.6 0.6 2.1 1 0.9 2.1 1.4 1 2.6 1.3 1.3 1.8 1.6 1.6 3.7 1.7 0.7 1.9 1.3 1.4 2.3 2 1.1 3.4

v 0.7 0.3 3.9 0.3 0.4 3.4 1 1 4.4 0.9 0.9 4.9 1 0.9 5.1 1.1 0.6 6.3 1.3 1.7 6.6 1 1.3 6.8 1.6 1.4 8.1

w 1 0.6 4.3 0.7 0.3 4.6 0.3 0.7 4.7 0.7 0.6 4.7 0.9 0.4 5 1.1 0.6 5 1.1 1.3 6.9 1.1 1.3 8.1 0.9 1.3 7.1

x 1 0.9 4.6 1.4 0.6 4.4 0.7 0.4 5.3 0.7 0.4 5.4 1.3 1.4 5.6 1.4 1.3 5.9 0.9 1.1 6.6 1.4 1.7 7.4 1.7 1 9.3

y 0.6 0.3 3.4 0.9 0.9 3.1 0.7 0.9 3.3 0.4 0.7 3.7 0.7 0.6 4.1 1.6 1.3 4.4 1.4 1.6 4.9 1.6 1.3 5.9 2 1.4 7.3

z 0.7 1.6 4 0.6 1.7 3.9 0.7 0.4 2.3 0.7 0.7 3.9 1.1 0.9 3.6 0.7 1 3.3 1.9 1.4 4.8 1.3 2.1 5.9 2 2.3 7.7

0 0.7 0.4 2.1 0.4 0.6 0.9 0.3 0.9 1.6 0.9 0.7 1.7 0.9 0.9 2.4 1.3 0.9 1 0.9 0.6 2 1 0.9 3.6 1.1 1.3 3.6

1 5.4 1.6 2.4 5.7 1.6 3.1 5.7 2 2.1 4.6 1.9 3.1 4.9 2.1 3 6.9 2.3 3.9 6.4 2.7 4 8.6 2.3 2.7 6.9 3.3 5

2 0.4 0.7 0.6 0.7 0.9 0.9 1 1.7 0.9 0.9 1.4 0.4 1.4 1.4 1.9 1.4 2 1.6 1.7 1.6 2.9 1.7 2.3 2.1 1.9 2 3.7

3 1.1 0.4 1.3 0.7 0.3 0.7 1.1 0.7 1.3 0.6 0.9 1.6 1.3 1.1 2.1 1.4 0.7 1 1.3 0.6 2.7 1.7 0.7 1.4 0.9 0.9 3.1

4 1.3 0.6 1 0.9 0.7 0.7 1.1 0.6 1.1 0.9 1.1 1.1 1.4 1 1.1 1.3 1.7 1.4 2 1.3 1.1 1.4 1.7 0.9 1.9 1.4 3.9

5 0.7 0.3 0.7 1.1 0.4 1.3 1 0.7 1.4 1.3 0.9 1.1 1 0.4 1.9 1.4 0.6 1.3 2.1 0.8 2 1.6 0.1 2.3 1.9 1.1 2.3

6 0.9 0.6 1.4 1 0.6 0.9 1 0.9 1.6 1.3 0.7 1.1 1.4 0.7 2 1.4 0.7 1.4 2.1 1 3.4 2.6 1 2.4 1.1 1.1 4

7 1.7 0.9 1.6 1.3 0.7 0.7 2.4 0.9 1.3 1.6 0.9 1.3 2.4 1 1.4 1.3 0.7 1 1.9 1.1 2.1 2.4 0.9 2.4 2.6 1.6 1.7

8 0.6 0.1 1.7 0.6 0.3 1.3 0.4 0.9 1.6 0.9 0.3 1.6 1.1 0.6 2.9 0.7 0.4 1.7 1 0.9 2.4 1.6 1 3.1 1 1.9 4.4

9 0.9 0.9 1 0.7 0.7 1.6 0.7 0.9 1.7 1 0.9 2 0.7 1 2.4 0.7 1.1 1.7 1.2 1.7 2.3 1 1.4 2.9 1.6 1.6 3.2

$ 0.4 0.1 2.6 0.7 0.6 3.4 0.7 0.3 2.6 0.7 0.6 4 2.1 0.6 4.1 1 0.3 4.1 1.1 0.9 4.9 1.1 0.7 5.3 0.7 0.7 8

£ 0.4 0.9 2.6 0.9 1.1 3.4 0.6 1.6 2.6 0.7 0.9 3.6 0.9 1.1 3.7 1 1.6 3.7 0.6 1.7 4.9 1 1.4 5.4 1.6 2.3 6

; 0.3 1 2.1 2.3 1.1 4.3 2.4 1.1 3 2.9 1.6 4.4 2.4 1.6 5.4 1.7 1.3 5 2.3 2.9 5.6 3.7 2.3 6 3.9 3.4 7.6

. 2.1 1.3 7.3 3.1 2.7 6.3 2.1 2.6 6 3.1 3.9 10 2.6 5 9.9 5 4.6 12 3.4 4.3 12 6.6 4.6 11 7.9 11 24

, 2.9 1 6.9 1.3 1.1 6.1 1.7 1.1 7.7 2.1 1.7 8.3 3.1 0.9 7.9 2.6 1.9 9.9 4.6 2.7 11 4.1 2.1 12 6.7 1.9 11

+ 0.6 0.3 1.4 0.6 0.7 1.3 0.3 0.7 2.1 0.7 0.7 0.7 0.9 0.9 0.7 0.6 0.6 1.9 0.6 1 2.1 0.7 1.7 1.7 1 2 1.9

- 1 2.3 3 1.1 2.1 3.3 1 4 6.9 1.1 5 11 1.3 4 4.1 1.1 3.4 5.4 1.4 5.7 11 1.7 6.5 5.1 2.9 10 12

* 0.7 1.3 6.1 0.7 1.1 7.3 1.1 0.9 6.7 0.9 1 8.1 1.4 0.6 9.7 1.7 0.7 9.9 1.9 1.6 10 2 1.6 14 1.6 2 14

% 0.4 0.4 4.3 0.4 0.4 4.7 0.7 0.9 4 0.4 0.6 5.1 0.4 0.9 4.9 0.4 0.3 6.2 1.1 1 6.6 0.7 1 7.3 1 0.9 8.7

Avr. 1 0.7 2.8 1.1 0.8 2.9 1.1 0.9 3 1.1 1 3.5 1.3 1 3.7 1.4 1 4.2 1.8 1.3 4.6 1.9 1.4 5.1 2.2 1.8 6.1

Table 5.7 : The Influence of Print-Scan Operation on the Centroids and the Rates of Black Pixels of 71 Characters Written in 9 Different Font Sizes


114

The average influence on the values R, X, and Y of the characters is shown in

Table (5.8) and Figure (5.4).

24 22 20 18 16 14 12 10 8

X 1 1.1 1.1 1.1 1.3 1.4 1.8 1.9 2.2

Y 0.7 0.8 0.9 1 1 1 1.3 1.4 1.8

R 2.8 2.9 3 3.5 3.7 4.3 4.6 5.1 6.1

Avr. 1.5 1.6 1.7 1.9 2 2.2 2.6 2.8 3.4

Table 5.8 : The Average Influence of Print-Scan Operation on the Centroids and Rates of Black Pixels of Characters Written in 9 Different Font Styles

Figure 5.4: The Average Influence of Print-Scan Operation on the Centroids and Rates of Black Pixels of Characters Written in 9 Different Font Styles

It is noticeable that characters with smaller font size are more affected by D⁄A

conversion than characters with larger font size. The difference rates are increasing

gradually when the font size is getting smaller. The print-scan conversion causes

additional noise around the inner and outer borders of characters in the scanned

image. The added or removed black pixels can significantly increase or decrease the

rate of black pixels and shift the centroid point (if the noise is not distributed evenly

0

1

2

3

4

5

6

7

24 22 20 18 16 14 12 10 8

X

Y

R

Avr.

Size of font

Influ

ence

rat

e


115

to the character image) in smaller font characters more than other characters with

larger fonts.

Therefore, the amount of noise makes a considerable difference in small font-

size characters while it makes less when added to larger size character images. This

explains the reason why large bold fonts are more often used in real life applications

of document verification as shown in the sample RYANAIR boarding pass in

Appendix-D where the font of the protected information such as the flying date and

the passenger name in this e-ticket is written in font sizes 12 and 25 and Verdana

Bold font.

Due to this diversity in the centroid position and rates of black pixels of different

fonts, Method 1 has to be seen as content dependent. Therefore, different thresholds

must be pre-set for different document images depending on the font size and style

of the textual image. The determined thresholds have to be a part of the

preservative data attached to the document and used during verification. The

experimental results of the verification system of Method 1 are given in the next

section.

5.2.2 Results of the Verification System of Method 1

Characters in document images can become connected to each other after a

print-scan operation. Therefore, it is better to divide each line into a number of parts

as previously suggested in section (4.7) to avoid the high rate of false-positive errors

type caused by character connections with a resulting rejection of authentic

documents having connected characters. Two sample document images divided by

using Method 1 are shown in Appendix A.


116

The 55 test documents have been printed and scanned without intentional

modifications. Also, each copy of the same 55 documents has been forged twice by

different people and scanned to the computer. The 55 unaltered and 110 forged

document images were then used to test the verification system of Method 1. Table

(5.9) show the results of the verification system of Method 1 on the above-

mentioned document images.

Image # Scanned Forged1 Forged2 Image # Scanned Forged1 Forged2 1 Rejected Rejected Rejected 29 Rejected Rejected Rejected 2 Verified Rejected Rejected 30 Verified Rejected Rejected 3 Verified Rejected Rejected 31 Rejected Rejected Rejected 4 Verified Rejected Rejected 32 Rejected Rejected Rejected 5 Rejected Rejected Rejected 33 Rejected Rejected Rejected 6 Verified Rejected Rejected 34 Rejected Rejected Rejected 7 Rejected Rejected Rejected 35 Rejected Rejected Rejected 8 Rejected Rejected Rejected 36 Rejected Rejected Rejected 9 Rejected Rejected Rejected 37 Rejected Rejected Rejected

10 Verified Rejected Rejected 38 Rejected Rejected Rejected 11 Rejected Rejected Rejected 39 Rejected Rejected Rejected 12 Verified Rejected Rejected 40 Rejected Rejected Rejected 13 Rejected Rejected Rejected 41 Rejected Rejected Rejected 14 Rejected Rejected Rejected 42 Verified Rejected Rejected 15 Rejected Rejected Rejected 43 Verified Rejected Rejected 16 Verified Rejected Rejected 44 Verified Rejected Rejected 17 Rejected Rejected Rejected 45 Verified Rejected Rejected 18 Verified Rejected Rejected 46 Verified Rejected Rejected 19 Verified Rejected Rejected 47 Verified Rejected Rejected 20 Rejected Rejected Rejected 48 Verified Rejected Rejected 21 Rejected Rejected Rejected 49 Verified Rejected Rejected 22 Rejected Rejected Rejected 50 Verified Rejected Rejected 23 Verified Rejected Rejected 51 Verified Rejected Rejected 24 Rejected Rejected Accepted 52 Verified Rejected Rejected 25 Rejected Rejected Rejected 53 Verified Rejected Rejected 26 Verified Rejected Rejected 54 Verified Rejected Rejected 27 Verified Rejected Rejected 55 Verified Rejected Rejected 28 Rejected Rejected Rejected

Table 5.9: The Experimental Results of Method 1 Verificatoin System on 55 Unaltered and 100 Forged Scanned Documents

The verification system frequently failed to verify genuine scanned documents

because it was sensitive to the print-scan operation. Twenty eight out of 55 genuine

scanned documents have been rejected by the system. The failure of the verifier is

due to the fact that scanned images are subject to rotation, additional noise, and


117

skew. Those applied modifications can mislead the verifier in detecting lines in the

scanned images. The verifier can detect a wrong width size of each line in the

scanned image if there is any level of rotation.

Consequently, the lines will be divided in a different way from how they were

divided during the creation process. The number of generated sub-images from lines

will be not equal to the number stored in the attached preservative data. As a result,

the false positive error rate will be extremely high. In real life applications, innocent

clients can lose trust in any organization that uses a document verification system

which rejects their genuine documents.

On the other hand, the verifier can easily detect small a forgery in intentionally

altered documents even if the forged images are perfectly scanned without rotation

and noise by comparing the rates of black pixels and the centroid points of the

generated sub-images with those stored in the preservative data. Manipulating the

thresholds Tcentroid and Tblack can adjust the sensitivity of the verifier. The smaller

Tcentroid and Tblack, the more sensitive the verifier is. In the experiments, both

thresholds were set to 7. Forgery has been detected in 109 out of 110 altered

documents.

Only one document with a single forged character among 441 characters was

considered as a genuine copy during the verification process. The reason why the

verifier failed to detect forgery is that the number of added black pixels was

accidently equivalent to the removed black pixels which did not change the rate of

black pixels in the image. Also, the new distribution of black pixels in the forged

image did not make a significant change in the location of the centroid point.

Therefore, the modified image was accepted by the system.


118

The exact locations of the altered parts were detected in Method 1 but there was

no possibility to recover the original contents. located in Table (5.10) shows the

statistical error of the verification system of Method 1.

Document Condition

Scanned (unaltered) Forged

Test

Results

Verified True Positive 49.1% False Negative 0.9 %

Rejected False Positive 50.9% True Negative 99.1%

Table 5.10: The Statistical Error of the Verification System of Method 1

5.3 The Experimental Results of Method 2 The same 55 scanned and 110 forged document images previously used in

Method 1 were also used to test Method 2. The whole contents of the document

images were selected as counterfeiters’ areas of interest to be protected. The text

images were recursively divided into 4 parts from the centroid point in Method 2

using the quad tree method. Two sample document images divided by Method 2 are

shown in Appendix B.

Two thresholds (Tmargin and TS) are required to stop the recursive division process

and another threshold (Tdiff) is needed to compare between the rate of black pixels of

the divided sub-images of the test scanned document and the original image as

explained in section (4.3.1) and (4.3.2). Different threshold values have been set for

document images depending on the contents of the document. Table (5.11) shows

the values used for each test image and the experimental results of the verification

system in Method 2 on the 55 scanned and 110 forged document images.


119

Thresholds Matching Rates

Img# TS Tmargin Tdiff Scanned Forged 1 Forged 2 No. of Parts 1 20 20 2 100 98.4 96.9 64 2 25 20 3 100 40.9 27.3 22 3 20 20 1 100 17.2 94.8 58 4 20 20 1 100 96.6 89.7 58 5 20 20 6 100 76.1 98.5 67 6 25 20 0.5 100 78.6 71.4 28 7 20 20 0.5 100 45.7 65.2 46 8 20 20 2 100 94.2 67.3 52 9 20 20 3 100 25 42.2 64 10 20 20 3 100 92.2 21.9 64 11 30 20 0.5 100 68.8 93.8 16 12 25 20 1 100 96.8 74.2 31 13 20 20 2 100 60.9 95.3 64 14 20 20 2 100 91 88.1 67 15 30 20 1 100 87.5 0 16 16 30 20 0.5 100 68.8 0 16 17 20 20 0.5 100 70.3 62.5 64 18 20 20 1 100 89.1 90.6 64 19 30 20 2 100 93.8 6.25 16 20 20 20 2 100 95.3 81.3 64 21 20 20 1 100 93.8 95.3 64 22 20 20 1 100 98.4 90.6 64 23 20 20 0.5 100 53.1 67.2 64 24 20 20 1 100 95.3 95.3 64 25 20 20 1 100 89.1 82.8 64 26 20 20 1 100 90.6 93.8 64 27 30 20 0.5 81.25 37.5 68.8 16 28 20 20 1 100 59.4 96.9 64 29 20 20 0.5 100 84.2 63.2 19 30 20 20 2 100 94.7 100 19 31 30 20 2 100 37.5 75 16 32 30 20 1 100 75 93.8 16 33 10 20 3 100 97.2 99.6 250 34 10 20 4 100 94.5 96.9 256 35 20 20 4 100 79.7 56.3 64 36 20 20 1 100 87.5 81.3 64 37 20 20 6 100 96.9 98.4 64 38 20 20 3 100 79.7 98.4 64 39 20 20 5 100 92.2 79.7 64 40 20 20 4 100 96.9 48.4 64 41 20 20 1 100 93.8 90.6 64 42 10 20 3 100 98 98.4 250 43 20 20 1 100 92.2 98.4 64 44 20 20 1 100 70.3 92.2 64 45 20 20 2 100 69 91.4 58 46 20 20 0.5 100 65 55 40 47 20 20 0.5 100 72.5 70 40 48 30 20 0.5 100 37.5 31.3 16 49 20 20 1 100 88.2 94.1 34 50 20 20 2 100 57.7 63.5 52 51 20 20 0.5 100 35.9 85.9 64 52 20 20 0.5 100 64.5 83.9 31 53 20 20 0.5 100 50 53.1 64 54 20 20 0.5 100 53.8 32.7 52 55 25 20 2 100 75 100 52

Table 5.11 : The Experimental Results of the Verification System of Method 2 on Scanned and Forged Images


120

The value of the Tmargin threshold is static in the experiments for all the test

document images but different values have been set for the other two thresholds TS

and Tdiff in order to minimize the error rate and obtain optimal results in verification.

The number of sub-images resulted from each document is shown in the right

side column of the table above. The table has drawn attention to the fact that the

smaller value of TS threshold leads to more sub-images derived from the document

image as in images 33, 34, and 42. On the other hand, the number of sub-images is

smaller when TS value is higher as in images 11, 15, 16, 19, 31, 32, and 48.

Among the 55 test scanned document images used in experiments, the verifier

succeeded in authenticating 98.2% of the documents after print-scan operation while

1.8% of the unaltered scanned documents were rejected by the verifier. The only

one unauthenticated out of 55 documents represents the false-positive (Type-α)

error rate in the system.

The verification system of Method 2 has successfully detected alterations in 108

out of 110 test forged images, the true-negative rate of counterfeit detection is

98.2%. On the other hand, the verifier failed to detect forgery in two forged text

images out of 110 images. This failure caused a 1.8% false-negative (i.e. Type-β)

error rate. The statistical error in the verification system of Method 2 is shown in

Table (5.12).

Document Condition


Test

Results

Verified True Positive 98.2% False Negative 1.8%


Table 5.12: The Statistical Error of the Test Results of Method 2


121

It is essential to mention that the two forged images that have been verified

and the unaltered image which has been rejected by the verifer have more white

areas than the other images which have been successfully authenticated or rejected.

The reason behind this failure is that some sub-images may have a small part of

a character located on the border while the rest of the sub-image is totally white. No

significant change occurs in the rate of black pixels of this sub-image when the

centroid in the parent image of this sub-image is shifted by a single bit. Also, any

added or removed black pixels to the text of this sub-image would not be able to

make a noticeble difference in the rate of the black pixels over the whole sub-image

which leads to a false-negative error. On this basis, it may be inferred that images

with extensive white parts can cause failure in the verification system. Therefore, it

is preferable to select areas of interest containing texts with few white areas.

5.4 The Experimental Results of the Extended Version of Method 2

The same test document images were also used to test the reliability and

efficiency of the modified version of Method 2 which recursively divides each image

into 2 parts either vertically or horizontally from its centroid point as discussed

earlier in section (4.4). An example of the division process of this method applied on

two sample images is shown in Appendix C. The experimental results of the

verification system of this method on the test documents are shown in Table (5.13).

Unlike in Method 2 where value of the value of Tmargin threshold was static, different

values for the three thresholds TS, Tmargin, and Tdiff were set to test each document

depending on the contents of the document.


122

Thresholds Matching Rates

Img# TS Tmargin Tdiff Scanned Forged 1 Forged 2 No. of Parts 1 0.2 0.2 0.5 100 50 100 8 2 0.25 0.2 2 100 20 20 10 3 0.25 0.25 2 100 9.1 63.6 11 4 0.2 0.2 1 100 95.2 95.2 21 5 0.25 0.2 1 100 0 30 10 6 0.25 0.25 0.5 100 81.2 62.5 16 7 0.25 0.25 0.5 91. 7 75 75 12 8 0.2 0.2 1 100 Forged Forged 22 9 0.2 0.2 3 100 33.3 46.7 30 10 0.2 0.2 4 Rejected Forged Forged 28 11 0.2 0.2 5 89.3 64.3 78.6 28 12 0.2 0.2 1 100 64.3 89.3 28 13 0.2 0.2 1 100 75 100 28 14 0.2 0.2 1 100 81.25 62.5 16 15 0.25 0.2 1 100 66.7 0 9 16 0.2 0.2 1 100 100 0 28 17 0.2 0.2 1 100 63.3 86. 7 30 18 0.2 0.2 1 100 65 55 20 19 0.2 0.2 0.5 100 13.8 65.5 29 20 0.2 0.2 1 100 96.4 85.7 28 21 0.2 0.2 1 100 100 86. 7 30 22 0.2 0.2 1 100 96.4 92.9 28 23 0.2 0.2 1 100 67.9 100 28 24 0.2 0.2 0.5 96.9 62.5 46.9 32 25 0.2 0.2 1 100 96.2 30.8 26 26 0.2 0.2 1 100 92 68 25 27 0.2 0.2 1 100 100 88 25 28 0.2 0.2 1 100 90 73.3 30 29 0.2 0.2 1 100 81.8 72.7 11 30 0.2 0.2 1 100 Forged Forged 13 31 0.2 0.2 1 100 75 75 12 32 0.2 0.2 1 100 53.6 92.9 28 33 0.2 0.2 1 100 97.1 76.5 34 34 0.2 0.2 1 100 59.4 81.25 32 35 0.2 0.2 1 22.6 32.3 16.1 31 36 0.2 0.2 1 100 93.1 82.8 29 37 0.2 0.2 1 100 3.6 Forged 28 38 0.2 0.2 1 100 71.4 71.4 28 39 0.2 0.2 1 100 73.3 73.3 30 40 0.2 0.2 1 100 23.3 0 30 41 0.2 0.2 1 100 100 96.7 30 42 0.2 0.2 1 100 50 67.9 28 43 0.2 0.2 1 100 96.8 Forged 31 44 0.2 0.2 1 100 95.8 79.2 24 45 0.2 0.2 1 100 96.4 100 28 46 0.2 0.2 0.5 100 68.4 63.2 19 47 0.2 0.2 0.5 90 65 80 20 48 0.2 0.2 2 100 62.1 Forged 29 49 0.2 0.2 1 100 73.1 100 26 50 0.2 0.2 1 100 73.1 73.1 26 51 0.2 0.2 0.5 Rejected Forged Forged 18 52 0.2 0.2 1 100 53.8 92.3 13 53 0.2 0.2 1 100 57.1 71.4 28 54 0.2 0.2 1 100 65 55 20 55 0.2 0.2 1 100 12.5 12.5 8

Table 5.13: The experimental Results of the Verification System of the Extended Version of Method 2 on Scanned and Forged Images


123

The verification system of this method shows higher rates of error than the

verification system of Method 2 because it failed to detect forgery in 9 out of 110

altered documents. Also, it rejected 7 out of 55 unaltered scanned documents. The

errors values during verification are written in bold font in Table (5.13). The words

Rejected and Forged in the table means that the number of the sub-images

generated during the verification process are not equal to the number of the sub-

images derived during the creation process. Therefore, the scanned image has been

considered as a forgery. The statistical error in the verification system of the

extended version of Method 2 is shown in Table (5.14).

Document Condition


Test

Results

Verified True Positive 87.3% False Negative 8.2%


Table 5.14: The Statistical Error in the Verification System of the Extended Version of Method 2

5.5 A Comparison between the Results of the Proposed Methods

As the same test document images were used to test the verification system of

all the proposed methods in this thesis, it is essential to compare the results to

identify the most efficient and reliable algorithm in detecting significant changes and

verifying scanned document images. Table (5.15) shows the average rates of the

successfully authenticated rejected document images by the verification system of

the three proposed methods after the print-scan operation.


124

Method 1 Method 2 The Extended version of Method 2

55 Unaltered scanned document images 49.1% 98.2% 87.3%

110 Forged scanned document images 99.1% 98.2% 91.8%

Average 74.1% 98.2% 89.55%

Table 5.15: The Average Rates of Successfully Verified and Rejected Documents by the Three Proposed Methods After Print-Scan Operation

The table above shows that Method 2 is more accurate and reliable for detecting

alterations in forged document as well as for verifying genuine scanned images as

this method has less sensitivity to the print scan operation than the other two

methods. Method 1 is the most sensitive and therefore it failed to verify more than

half of the unaltered scanned documents. On the other hand, it has successfully

rejected the vast majority of the forged document due its sensitivity. Overall, it is

better to rely on Method 2 as it shows the highest average rate of accuracy in

verification.

5.6 The Influence of Rotation on the Verification Process

A sample document has been rotated by 0.75 degrees clockwise and counter-

clockwise with 0.05 intervals and each rotated image has been compared with the

original document. The matching rates of the rotated and original images in the

verification system of Method 2 are given in Figure (5.5) and Table (5.16).


125

Figure 5.5: The Rotation Influence on Document Verification

Clockwise (CW) Counter Clockwise (CCW)

Rotation Degree

Matching Rate

Rotation Degree

Matching Rate

0.05 100% -0.05 100%

0.1 100% -0.1 100%

0.15 99.09% -0.15 100%

0.2 99.09% -0.2 99.09%

0.25 99.09% -0.25 99.09%

0.3 97.24% -0.3 98.16%

0.35 98.16% -0.35 98.16%

0.4 95.41% -0.4 97.24%

0.45 96.33% -0.45 98.16%

0.5 96.33% -0.5 96.33%

0.55 95.45% -0.55 95.45%

0.6 95.45% -0.6 96.33%

0.65 96.33% -0.65 96.33%

0.7 93.39% -0.7 96.33%

0.75 93.39% -0.75 83.48%

Table 5.16: The Rotation Influence on Document Verification

5.7 Invisible Data Hiding The preservative data of a document can be invisibly embedded into the

document image itself by using any of the invisible data-hiding methods for binary

75.00%

80.00%

85.00%

90.00%

95.00%

100.00%

-0.7

5-0

.7-0

.65

-0.6

-0.5

5-0

.5-0

.45

-0.4

-0.3

5-0

.3-0

.25

-0.2

-0.1

5-0

.1-0

.05 0

0.05 0.1

0.15 0.2

0.25 0.3

0.35 0.4

0.45 0.5

0.55 0.6

0.65 0.7

0.75 Rotation Degree

Matching Rate


126

images provided that they do not visually affect the binary images to the extent of

leading to different preservative data. The dots-shift data-hiding technique can be

implemented to embed a binary stream of data into a bi-level document image. The

main idea of the dots-shifting technique is to shift the dots of the letters (i & j) in the

document image very slightly (up, down, left, right, or diagonally) to encode 3 bits

as a maximum amount of data per dot as shown in Figure (5.6).

Figure 5.6 : An Example of Dot Shifting Technique to Embed 3 Bits of Data by Shifting the Dot in the Letter (i)

Theoretically, the proposed technique does not cause noticeable irregularities on

the host document image. To extract the hidden data, the locations of shifted dots

must be determined in order to detect the horizontal direction of shifting (left or

right). Also, the distance between the centroid of the dot and the character which

this dot belongs to must be calculated to find out the vertical shifting direction (up or

down).

The Data Hiding Capacity (DHC) depends on the quantity of the available dotted

letters (i & j) in documents written in English language. In a sample English text

book contains 2,188,153 words in 4466 pages (i.e. 489.98 words⁄page), the number

of the letters (i and j) has been computed. The percentages of frequencies of (i & j)

letters per 1000 words and per page have been calculated as shown in Table (5.17).


127

No. of words & pages

No. of (i & j) Words = 2,188,153 Pages = 4,466

(i) = 569,016 260 (i ⁄ 1000 words) 127.4 (i ⁄ page)

(j) = 9572 4.3 (j ⁄ 1000 words) 2.143 (j ⁄ page)

Table 5.17 : The Percentages of Frequencies of (i & j) Letters per 1000 Words and Per Page in a Sample (2,188,153 words) Text Book

The data hiding capacity (bits ⁄page) for a text can be calculated by equation

(6.1) below:

Data Hiding Capacity (bits ⁄page) = 3 × [(No. of i) + (No. of j)] ... equation (6.1)

The average DHC (bits ⁄page) for that sample text book is:

DHC = 3 × [127.4 + 2.143]

= 388 bits (that is 48 ASCII chars)

The capacity differs from language to another because texts in some languages

have more dots than other languages, for instance there are more dotted letters in

the Arabic language than those in English. On the other hand, some languages like

Chinese or Hebrew do not have dotted letters which makes this technique not

applicable for those languages. Therefore, this data hiding techniques would be

useful only for some languages. More experiments are needed to prove this

hypothesis.

Chapter 6: Conclusions and Future Research

128

Chapter Six: Conclusions and Future Research

6.1 Recommendations to Improve the System

The methods proposed in this research can be developed by adding more

features or by replacing some algorithms with others. This chapter sheds light on

techniques that if they were applied, they could possibly enhance the performance,

reliability, and security level of the verification system. The main aim of making

these changes would be to make the verification system more reliable in detecting

forgery and in authenticating original copies. Also, the system could be improved by

employing an invisible data-hiding method to embed the preservative data. Various

suggestions and recommendations for developing the system are given in this

section. One or more of the techniques listed below could be added to the document

creation and validation algorithms in order to enhance the verification method.

6.1.1 Noise Removal

Noise can occur in documents when they are printed, sent, and rescanned into

the computer because of factors such as inconsistencies of printers, unclean flatbed

glass of scanners, humidity, or the exposure of hard copy to direct sunlight for long

time. Also, the careless handling of documents by their owners can be another cause

of noise in documents, for example if documents become slightly torn or bent. If the

amount of the noise is significant, the verifier would count this noise as additional

unwanted elements in the scanned document thereby leading to false detection of

tampering and the possiblity that the document will be considered as a forgery and

rejected by the system.


129

Therefore, it is essential to apply a noise removal method to binary images that

recognize the difference between the original text of the document and the added

noise followed by elimination of this undesirable noise (Agrawal & Doermann, 2009).

The challenge in the noise removal is that the real text of the imaged document

should not be damaged, no matter what the size of the text. In this research, Method

1 used a simple noise removal technique to eliminate isolated dots or lines from the

document image. However, a more accurate and efficient noise reduction technique

is recommended for use in the future.

6.1.2 De-skewing

Documents can be rotated clockwise or anticlockwise during printing and

scanning operations. In printers, if the paper tray inaccurately feeds the printer, the

text lines will be irregularly oblique on the printed paper. On the other hand, if the

document is not well placed in the flatbed glass of a scanner, the scanned text will be

slanted in the document image. This rotation in the printed/scanned document can

deceive the verifier as discussed earlier in section (5.6). As a result, the document

might be considered as a falsified paper when actually it is authentic. For that

reason, it is necessary to identify the orientation of the scanned image and to

compute its degree of rotation in order to deskew and straighten the text lines before

validating the document (Baird, 1995).

Indeed, unaltered documents are supposed to be verifiable despite the

unintentional rotation that can happen during scanning. However, the system often

fails to verify rotated documents with genuine contents because binary document

images are highly sensitive to any degree of rotation. The rotation causes shifting in

centroid positions. As a result, the image will be divided in different ways creating

dissimilar sub-images compared with the original. Consequently, the rotated image

will be considered as a false document which makes the verification system


130

vulnerable and causes a false-positive error. Also, the failure of Method 1 to verify

unaltered scanned images is because this method is very sensitive to rotation, as

discussed in Chapter Five.

Due to the substantial impact of rotation and skewing on the authentication

process, it has been vital to find the rotation degree in order to regulate the rotated

image and restore its original form before verifying it. Images with borders or with

plain vertical/horizontal lines can be adjusted simply by computing the rotational

degree. However, documents with no straight lines need other adjustment

techniques. To avoid the false-positive-error caused by rotation, one of the deskew

techniques proposed in the literature should be used to align scanned documents

prior to the verification process.

6.1.3 Optical Character Recognition (OCR)

If the actual formatted text of a document is used as preservative data and

embedded in the attached barcode, the OCR technique can be used to convert the

document image into formatted text (Wang & Bunke, 1997) and this text can be

compared with the preservative data previously embedded in the document itself.

The use of OCR could help to avoid the weaknesses of Method 1 in recognizing ASCII

characters as previously discussed in section 4.6.

However, OCR has limitations because it can be used only for text documents

that do not contain any images and graphs. Also, multilingual documents need an

OCR system which is able to recognize different languages. Therefore, the reliability

of the verification system will depend on the efficiency of the OCR. The failure of the

OCR to detect the text correctly could lead to false-positive errors. As a result, the

OCR software has to be carefully chosen and tested on the text used in typical

documents.


131

6.1.4 Data Compression

The size of the preservative data extracted from the original document can be

reduced using a data compression technique to embed the maximum amount of

information about the original document in the barcode in order to increase the

reliability of the verification system.

The range of black pixels rates needed to represent the binary sub-images can

be computed. Each value might be represented in fewer bits if the values were

reduced by scaling. For example, if the range is (R: min to max), it can be re-scaled

to (R‘:0..max-min) in order to diminish the size of the preservative data. Also, the

values of the calculated centroids can be ranged and encoded. Therefore, fewer bits

will be needed to represent the preservative data. Popular data compression

algorithms such as Run Length Encoding (RLE) can be used to reduce the size of the

data (Salomon, 2007).

6.1.5 Data Encryption

As discussed in Chapter 4, the preservative data are information about some

areas in the binary document image which may be subject to forgery. This

information could be the location of centroids, rate of black pixels, or a combination

of both. If counterfeiters know what the representation of the area of interest is,

they could re-generate the preservative data after altering the document image.

Then, this data can be stored in a barcode and attached to the document. The

verification system will authenticate this forged document because its details match

the data extracted from the barcode.

Therefore, it is recommended to encrypt the preservative data before saving that

data in the barcode. One of the most popular encryption algorithms such as DES


132

(Data Encryption Standard) or RSA which is proposed by Ron Rivest, Adi Shamir,

and Leonard Adleman, or any other secure algorithm can be used. A public/private

keys must be used during encryption where only the creator knows the encryption

secret key. On the other hand, the verification system has the decryption key which

is used to convert the ciphered data into plain preservative data (Mollin, 2007;

Konheim, 2007). If counterfeiters intend to produce a valid fake document, they

need to know not only the format of the preservative data but also the encryption

keys. Therefore, the use of a data encryption algorithm makes it harder for

counterfeiters to create modified documents which can be verified by the

authentication system.

6.1.6 Auto Selection of Data Embedding

The system could be modified to select a suitable data embedding technique

automatically if there were more than one technique available. This can be decided

depending on the size of the preservative data and the hiding capacity of the host in

each existing technique. Also, the selection priority could vary depending on the

system requirements. For example, if the 2D data matrix barcode and the dot

shifting discussed in section 6.2.5 were the two available embedding methods, the

system should be able to choose the one with higher data capacity to hold the

preservative data. The Dot-shifting method should be chosen only if there are

enough dots in the host document image to carry the preservative data. Otherwise,

the data would be encoded in a 2D barcode as shown in Figure (6.1).


133

Figure 6.1: A Flowchart for the Auto-Selection of Data Embedding Technique

6.2 Conclusions

The print-scan operation has a substantial impact on the contents of documents.

Given this fact, scanned images cannot be exact copies of their originals because of

the potential noise and rotations that can be introduced into the document during

print or scan operations. The proposed methods have been designed to verify

unaltered scanned document images as well as to detect the potential intentional

alternations in ordinary paper documents. The contribution of the proposed system is

the combination of using centroids, quadtree, and 2D Data Matrix barcodes that

generate a novel system to verify scanned documents efficiently. The unique use of

the above mentioned combination has created efficient methods to detect forgery

accurately in document images. The superior ability of the methods proposed in this

research to verify monochrome document images explains why those methods have

a higher level of accuracy in detection than other techniques previously proposed in

literature. The rates of black pixels and the centroid points were used as the

preservative data in the three methods. Those methods have shown different levels

Compute the number of dotted letters in

the host image The preservative data

Embed the data in a barcode

Use Dots-Shifting Data Hiding Technique

Is

DHC of the host enough to

hold the preservative

data?


134

of sensitivity to D/A conversion by low-priced scanners. Fifty five unaltered scanned

documents and 110 slightly altered documents were used in experiments to test the

verification systems of the proposed methods. The majority of the scanned images

have successfully been verified while the majority of the forged documents have

been rejected. Method 2 has a high level of reliability for distinguishing between

malicious and accidental modifications in the scanned documents. It is the least

sensitive method for artefacts introduced by the print-scan operation because it has

effectively managed to verify unaltered, but scanned, documents. Method 1 is

sensitive to D/A conversion but is also effective in detecting small intentional

alterations in forged documents. The locations of altered parts have also been

detected by the proposed methods.

Appendices

135

Appendices

Appendices

136

Appendix A

Appendix A: Two Sample Document Images Divided by Method 1

Appendices

137

Appendix B

Appendix B: Two Sample Document Images Divided by Method 2

The depth levels of image division are shown in different colours. Level 1:Green, Level 2:Red, Level 3:Blue, and Level 4:Purple.

Appendices

138

Appendix C

Appendix C: Two Sample Document Images Divided by The Extended Version of Method 2

The depth levels of image division are shown in different colours. Level 1:Green, Level 2:Red, Level

3:Blue, Level 4:Purple, Level 5: Silver, Level 6:Yellow, Level 7:Black, and Level 8:Aqua.

Appendices

139

Appendix D

Appendix D: A Sample RYANAIR Flight Boarding Pass

References

140

List of References

Agrawal, M. & Doermann, D. (2009), "Clutter Noise Removal in Binary

Document Images", International Conference on Document Analysis

and Recognition (ICDAR '09).

Alattar A. M. and Alattar O M. (2004) “Watermarking electronic text

documents containing justified paragraphs and irregular line spacing”.

Security, Steganography, and Watermarking of Multimedia Contents

2004: pp. 685-695.

Arnold, M. (2003). Techniques and Applications of Digital Watermarking and

Content Protection. Norwood, MA: Artech House, Incorporated.

Baird, H.S. (1995), "The skew angle of printed documents". In Document

image analysis, IEEE Computer Society Press, Los Alamitos, CA, USA.

Berg, M. D. & Cheong, O. (2008), Computational Geometry: Algorithms and

Applications, 3rd ed, Springer-Verlag, Heidelberg.

Bourke, P. (1988), Calculating the Area and Centroid of A Polygon. Available

from: The University of Western Australia, Web site:

http://local.wasp.uwa.edu.au/~pbourke/geometry/polyarea/

[Accessed: March 11, 2009].

Brown, L. (1993), The New Shorter Oxford English Dictionary on Historical

Principles, 2nd ed, Oxford University Press Inc. New York, USA.

Chen, M., Memon, N. & Wong, E. K. (2004). Data Hiding in Document

Images. In Lu C. Multimedia Security: Steganography and Digital

Watermarking Techniques for Protection of Intellectual Property (pp.

http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/a/Alattar:Osama_M=.html�

http://www.informatik.uni-trier.de/~ley/db/conf/sswmc/sswmc2004.html#AlattarA04�

http://www.informatik.uni-trier.de/~ley/db/conf/sswmc/sswmc2004.html#AlattarA04�

References

141

231-247). Hershey, PA, USA: Ideal Group Publishing.

Chow, S. Serinken, N. & Seymour, S. (1993), Security Technology 1993,

Institute of Electrical and Electronics Engineers 1993 International

Carnahan Conference on Security Technology, Proceedings, vol. 6, pp.

11-14.

Chung, K.L. & Chen, P.C. (2005), "An efficient algorithm for computing

moments on a block representation of a greyscale image", Pattern

Recognition , vol. 38, no. 12, pp. 2578-2586.

Dai, M. Baylou, P. & Najim, M. (1992), "An efficient algorithm for

computation of shape moments from run-length codes or chain

codes", Pattern Recognition , vol. 25, no. 10, pp. 1119-1128.

Digimarc (2009), Digimarc (DMRC) Enriched Media, Enriching Lives.

Available from: https://www.digimarc.com [Accessed: February 12,

2009].

Easton, V.J. & McColl, J.H. (1997), Statistics Glossary v1.1. Available from:

University of Glasgow, Web site:

http://www.stats.gla.ac.uk/steps/glossary/index.html [Accessed:

September 19, 2009].

Finkel, R. A. and Bentley, J. L. (1974), "Quad trees a data structure for

retrieval on composite keys ", Acta Informatica, vol. 4, no. 1, pp. 1-9.

Flusser, J. (1998), "Fast calculation of geometric moments of binary

images", in: Proc. 22nd OAGM’98 Workshop Pattern Recognition

Medical Computer Vision, vol. no. pp. 265–274.

http://www.stats.gla.ac.uk/steps/glossary/index.html�

References

142

Fridrich J (1998b), "Methods for Detecting Changes in Digital Images", Proc.

of the 6th IEEE International Workshop on Intelligent Signal Processing

and Communication Systems, 5-6 Nov. 1998, Melbourne, Australia,

pp.173-177.

Fridrich J. (1998a), "Image Watermarking for Tamper Detection ", Proc.

IEEE International Conf. on Image Processing, ICIP-98, Vol.2, pp.404-

408.

Fridrich J. (1999a), "Methods for tamper detection in digital images," Proc.

ACM Workshop on Multimedia and Security, pp. 19--23, Orlando, FL,

October 30--31, 1999.

Fridrich J. (1999b), A Hybrid Watermark for Tamper Detection in Digital

Images, ISSPA'99 Conf., Brisbane, Australia, August 22-25, 1999.

Fridrich J., Goljan M., and Baldoza A. (2000), "New fragile authentication

watermark for images," in Proceedings of IEEE International

Conference on Image Processing, (Vancouver, Canada), September

10-13, 2000.

Fridrich, J. & Goljan, M. (1999). Protection of Digital Images Using Self

Embedding. Symposium on Content Security and Data Hiding in Digital

Media.

Fridrich, J. Goljan, M. & Lisonek, P. (2005), "Writing on Wet Paper", IEEE

Trans. on Signal Processing, vol. 53, no. 2, pp. 3923- 3935.

Furht, B., Muharemagic, E. & Socek, D. (2005). Multimedia Encryption and

Watermarking. New York: Springer.

Gou, H. & Wu, M. (2007), "Improving Embedding Payload in Binary Images

with "Super-Pixels"", Image Processing, 2007. ICIP 2007. IEEE

References

143

International Conference on, vol. 3, pp. 277- 280.

Han, S. Y., Jung E. H., and Cho S. Y. (2005). A Robust Digital Watermarking

Adopting 2D Barcode. Computer Analysis of Images and Patterns: 11th

International Conference, CAIP 2005, Versailles, France, September 5-

8, 2005. Proceedings, 3691 / 2005 p 717-723.

Har-Peled, S. (2008), Geometric Approximation Algorithms. Available from:

University of Illinois, Department of Computer Science Web site:

http://valis.cs.uiuc.edu/~sariel/teach/notes/aprx/ [Accessed: March

13, 2009].

Hsieh, M. & Tseng, D. (2004), "Perceptual Digital Watermarking for Image

Authentication in Electronic Commerce", Electronic Commerce

Research, vol. 4, no. 1-2, pp. 157-170.

Katzenbeisser, S. (2000), Information Hiding Techniques for Steganography

and Digital Watermarking, Artech House Books, Boston, MA.

Kim H. Y. and Queiroz R. L (2004a). ‘A public-key authentication

watermarking for binary images’. ICIP 2004: 3459-3462.

Kim, H. Y. and Queiroz, R. L. (2004b), “Alteration-Locating Authentication

Watermarking for Binary Images,” Int. Workshop on Digital

Watermarking 2004, (Seoul), Lecture Notes in Computer Science 3304,

pp. 125-136, 2004.

Kim, H.Y. & Afif, A. (2003), "Secure Authentication Watermarking for Binary

Images ", XVI Brazilian Symposium on Computer Graphics and Image

Processing (SIBGRAPI'03), pp. 199.

Kim, H.Y. & Afif, A. (2004), "A Secure Authentication Watermarking for

References

144

Halftone and Binary Images", International Journal of Imaging

Systems and Technology, vol. 14, no. 4, pp. 147 - 152.

Kim, Y. & Oh, I. (2004). Watermarking text Document images using edge

direction histogram. Pattern Recognition Letters, 25 (11), 1243-1251.

Konheim, A. G. (2007), Computer Security and Cryptography, Wiley & Sons,

Inc., Hoboken, New Jersey.

Koschan, A. & Abidi, M. (2008), Digital Color Image Processing, John Wiley &

Sons, Inc. Hoboken, New Jersey.

Kwok, S.H. Yang, C.C. & Tam, K.Y. (2000), "Watermark Design Pattern for

Intellectual property Protection in Electronic Commerce Applications",

Proceedings of the 33rd Hawaii International Conference on System

Sciences, vol. 6, pp. 6038.

Lan, T. & Tawfik, A. H. (1999). Fraud Detection and Self Embedding.

Proceedings of the seventh ACM international conference on

Multimedia (part 2), 33 – 36.

Li, B.C. (1993), "A New Computation of Geometric Moments", Pattern

Recognition , vol. 26, no. 1, pp. 109-113.

Li, C. Lou, D. & Liu, J. (2003), "Image Integrity and Authenticity Verification

Via Content-based Watermarks and a Public Key Cryptosystem",

Journal of Chinese Institute of Electrical Engineering, vol. 10, no. 1,

pp. 99-106.

Lin, C. & Chang, S. (2001). SARI: self-authentication-and-recovery image

watermarking system. International Multimedia Conference, volume 9,

References

145

628 – 629.

Liu, H. Rui, W. & Huang, J. (2007), "Binary Image Authentication using

Zernike Moments", IEEE International Conference on Image

Processing, ICIP 2007. , vol. 1, pp. 385 - 388.

Low, S. H., Maxemchuk, N. F. & Lapone, A. M. (1998). Document

Identification for Copyright Protection Using Centroid Detection. IEEE

Transactions on Communications, 46 (3), 372-383.

Lu H, Kot, A.C. and Cheng J (2003); ‘Secure data hiding in binary document

images for authentication’, Circuits and Systems, 2003. ISCAS '03.

Proceedings of the 2003 International Symposium on, Volume 3, 25-28

May 2003, Pages: III-806 - III-809.

Mei Q. G., Wong E. K. and Memon N. D. (2001), ‘Data hiding in binary text

documents,’ in Proc. of SPIE Security and Watermarking of Multimedia

Contents III, Aug. 2001, vol. 4314, pp.369–375.

Mohanty, S.P. Ramakrishnan, K.R. & Kankanhalli, M. (1999), "A dual

watermarking technique for images", Proceedings of the seventh ACM

international conference on Multimedia, vol. 2, pp. 49 - 51.

Mollin, R. A (2007), An Introduction to Cryptography, 2nd ed, Chapman &

Hall/CRC, New York.

Onwuegbuzie, A.J. & Daniel, L. G. (2003), "Typology of Analytical and

Interpretational Errors in Quantitative and Qualitative Educational

Research", Current Issues in Education, Vol.6, No.2.

Pratap, R. & Ruina, A. (2009), "Center of mass and gravity". In Introduction

to Statics and Dynamics, Oxford University Press, Oxford.

References

146

Pratt, W. K. (2007), Digital Image Processing, 4th ed, John Wiley & Sons,

Inc. Los Altos, California.

Puhan, N.B. & Ho, A.T. (2005), "Binary Document Image Watermarking for

Secure Authentication Using Perceptual Modeling", Signal Processing

and Information Technology, 2005. Proceedings of the Fifth IEEE

International Symposium, vol. no. pp. 393 - 398.

QuadTree (2009), QuadTree. Available from:, Wikipedia, the free

encyclopedia Web site: http://en.wikipedia.org/wiki/Quadtree

[Accessed: June 29, 2009].

Rogers, T. & Link, T. (1998), Amazing Applications of Probability and

Statistics. Available from: Intuitor.Com, Web site:

http://www.intuitor.com/statistics/ [Accessed: September 19, 2009].

Salomon, D. (2007), Data Compression: the Complete Reference, 4th ed,

Springer, London.

Schneider, M. (1996), "A robust content based digital signature for image

authentication", International Conference on Image Processing,

Proceedings. vol. 3, pp. 227 - 230.

Sharkas, M. Elshafie, D. & Hamdy, N. (2005), "A Dual Digital-Image

Watermarking Technique", PROCEEDINGS OF WORLD ACADEMY OF

SCIENCE, ENGINEERING AND TECHNOLOGY, vol. 5, pp. 136-139.

Signum Technologies (2009), Digital Watermarking from Signum

Technologies. Available from: http://www.signumtech.com [Accessed:

February 12, 2009].

References

147

Sun, Q.B. Feng, P.R. & Deng, R. (2001), "An optical watermarking solution

for authenticating printeddocuments", Information Technology: Coding

and Computing, 2001. Proceedings. International Conference on, pp.

65 - 70.

Sutcu, Y. Coskun, B. & Sencar, H.T. (2007), "Tamper Detection Based on

Regularity of Wavelet Transform Coefficients", IEEE International

Conference on Image Processing. ICIP 2007., vol. 1, pp. 397 – 400.

Tan, C.L., Huang, W. & Yu, Z. (2002), "Imaged Document Text Retrieval

Without OCR", IEEE Transaction on Pattern Analysis and Machine

Intelligence, vol. 24, no. 6, pp. 838-844.

The IET (2005) “The 2D Data Matrix Barcode”, Computing and Control

Engineering. December 2005, Volume 16, Issue 6, p. 39.

Tsai, J. Huang, W. & Chen, C. (2007), "A Feature-Based Digital Image

Watermarking for Copyright Protection and Content Authentication",

IEEE International Conference on Image Processing, ICIP 2007, vol. 5,

pp. 469 - 472.

Wang, P. S. & Bunke, H.(1997), Handbook on Optical Character Recognition

and Document Image Analysis, World Scientific Publishing Co Pte Ltd,

London.

Wong, P.W. (1998), "A public key watermark for image verification and

authentication", International Conference on Image Processing, 1998.

ICIP 98. Proceedings, vol. 1, pp. 455-459.

Xiaoyi, Y. Yunhong, W. & Tieniu, T. (2004), "Steganalysis of Data Hiding in

References

148

Binary Images", Proceedings of the Pattern Recognition, 17th

International Conference on (ICPR'04), vol. 4, pp. 877 - 880.

Yang H. and Kot A. C. (2004) ‘Data hiding for bi-level documents using

smoothing technique’. ISCAS (5) 2004: 692-695.

Yeung, M. M. and Mintzer, F. C. (1997) "Invisible watermarking for image

verification," in Proc. International Conference on Image Processing,

vol. 2, (Washington, DC), pp. 680--683, October 1997.

Yu, L. Niu, X. & Sun, S. (2005), "Print-and-Scan Model and the

Watermarking Countermeasure", Image and Vision Computing, vol. 23,

pp. 807-814.

Zhao, J. & Koch, E. (1995), "Embedding Robust Labels into Images for

Copyright Protection", Proceedings of the International Congress on

Intellectual Property Rights for Specialized Information, Knowledge and

New Technologies, vol. pp. 242-251.

Zhu, B., Wu, J. & Kankanhalli, M. S. (2003). Print signatures for document

authentication. Conference on Computer and Communications Security

(Proceedings of the 10th ACM conference on Computer and

communications security), 145 - 154.

Date post:	18-Nov-2023
Category:	Documents
Upload:	nottingham
View:	0 times
Download:	0 times