1500 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. …hklee.kaist.ac.kr/publications/IEEE Trans on...

1500 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 8, DECEMBER 2008

Difference Expansion Based Reversible DataHiding Using Two Embedding Directions

Yongjian Hu, Member, IEEE, Heung-Kyu Lee, Kaiying Chen, and Jianwei Li

Abstract—Current difference-expansion (DE) embedding tech-niques perform one layer embedding in a difference image. They donot turn to the next difference image for another layer embeddingunless the current difference image has no expandable differencesleft. The obvious disadvantage of these techniques is that imagequality may have been severely degraded even before the later layerembedding begins because the previous layer embedding has usedup all expandable differences, including those with large magni-tude. Based on integer Haar wavelet transform, we propose a newDE embedding algorithm, which utilizes the horizontal as well asvertical difference images for data hiding. We introduce a dynam-ical expandable difference search and selection mechanism. Thismechanism gives even chances to small differences in two differ-ence images and effectively avoids the situation that the largestdifferences in the first difference image are used up while there isalmost no chance to embed in small differences of the second dif-ference image. We also present an improved histogram-based dif-ference selection and shifting scheme, which refines our algorithmand makes it resilient to different types of images. Compared withcurrent algorithms, the proposed algorithm often has better em-bedding capacity versus image quality performance. The advan-tage of our algorithm is more obvious near the embedding rate of0.5 bpp.

Index Terms—Data hiding, difference expansion embedding, in-teger Haar wavelet transform, reversible watermarking.

I. INTRODUCTION

R EVERSIBLE data hiding can provide the extracted dataas well as the losslessly recovered original image at the

receiver end. This property makes it very useful in the areaswhere image quality is strictly required though the images needthe embedded data (watermark) to protect their authenticity/in-tegrity or for some special purposes. The potential applicationsfor this technique cover authentication or content integrity veri-fication (e.g., [1], [2]), covert communication, and image/videocoding (e.g., [3], [4]).

In literature, quite a few reversible data hiding algorithmshave been proposed (e.g., [5]–[14]). Typical embedding strate-gies include modulo 256 addition in [1], lossless compression

Manuscript received September 12, 2007; revised July 28, 2008. Current ver-sion published December 10, 2008. This work was supported in part by BK21 project of KAIST, KOSEF Grant NRL Program R0A-2007-000-20023-0 ofMOST, and the NSF of China 60772115 and 60572140. The associate editorcoordinating the review of this manuscript and approving it for publication wasDr. Ishfaq Ahmad.

Y. Hu and H.-K. Lee are with the Department of Computer Science, KoreaAdvanced Institute of Science and Technology, Daejeon 305-701, Korea(e-mail: [email protected]).

Y. Hu, K. Chen and J. Li are with the College of Automation Science andEngineering, South China University of Technology, Guangzhou 510641, China(e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TMM.2008.2007341

in [5], [8] and [9], difference expansion in [6] and [7], and his-togram modification in [10] and [12]. Among them, DE embed-ding proposed by Tian in [6] has attracted a lot of attention forhigh embedding capacity. There are different variants or exten-sions of the DE method (e.g., [7], [9]). The latest extension isThodi et al.’s work [14], which has greatly improved the orig-inal DE method in both embedding capacity and image quality.

In general, DE methods are based on difference images ob-tained by using techniques derived from image coding. A differ-ence image may have smaller size than the original image dueto mathematical transformations. For example, when the integerHaar wavelet transform is performed in row direction, the ob-tained horizontal difference image and average image have thehalf size of the original image in that direction. It implies thatthe data hiding technique based on the difference image cannever exceed the embedding rate of 0.5 bpp in a single layerembedding. Tian pointed out this embedding capacity limit in[6]. To achieve larger embedding capacity than this limit, hesuggested multiple-layer embedding. For example, the secondlayer embedding would take place in the orthogonal direction,where the difference image is obtained by performing integerHaar wavelet transform on the embedded image in column di-rection. If the capacity of the two-layer embedding is still in-sufficient for the payload, a third layer embedding is needed.One performs integer Haar wavelet transform in row directionagain and repeats the embedding operation. Such a process con-tinues until the total embedding capacity is large enough for thepayload. However, multiple-layer embedding results in someunexpected problems. First, image quality (in terms of peaksignal-to-noise ratio (PSNR)) drops greatly after the first layerembedding due to the use of large differences. Second, the newdifference image has smaller embedding capacity than its prede-cessor. Each layer-embedding progressively decreases the cor-relation not only in the embedding directions but also of theneighborhood. Thus, multiple-embedding does not effectivelyexploit the correlation inherent in a neighborhood [14]. Third,the algorithm can not keep its behavior smoothly because eachlayer embedding has its own embedding capacity limit. Thesudden dip in the capacity versus distortion curve for the DEmethods around 0.5 bpp is the effect of multiple-layer embed-ding [14].

In this work, we aim at solving the mentioned problems.Based on integer Haar wavelet transform, we propose analgorithm that selects expandable differences under the sameselection threshold in two difference images and embeds thepayload in two orthogonal embedding directions. Furthermore,no matter in which difference image, our algorithm can alwaysgive priority to the use of small differences. This schemegreatly improves image quality. Our algorithm does not havethe original layer embedding capacity limit. The algorithm

1520-9210/$25.00 © 2008 IEEE

Authorized licensed use limited to: Korea Advanced Institute of Science and Technology. Downloaded on December 21, 2008 at 23:48 from IEEE Xplore. Restrictions apply.

HU et al.: DIFFERENCE EXPANSION BASED REVERSIBLE DATA HIDING USING TWO EMBEDDING DIRECTIONS 1501

performance is smooth and varies gradually with the change ofpayloads.

The rest of this paper is organized as follows. Section IIbriefly reviews the DE-based algorithms in [6] and [14]. Sec-tion III describes our reversible data hiding algorithm in detail.Section IV gives the experimental results of our algorithm. Wealso compare it with other current algorithms. In Section V, wedraw the conclusion.

II. REVIEW

This section briefly reviews the principle of DE embeddingand the typical algorithms in [6] and [14]. Suppose a pair ofimage pixel values , , their integer average

and difference are computed as

(1)

The inverse transform of (1) is

(2)

The reversible transforms (1) and (2) are called integer Haarwavelet transform in [6]. Apparently, and are the low-fre-quency and high-frequency signal components, respectively.Tian kept unchanged and embedded a binary bit into bythe following DE rule.

(3)

Tian defined expandable differences by (4) and exchangeabledifferences by (5). The constraint (4) ensures the inverse trans-form between (1) and (2).

(4)

(5)

where for an 8-bit image. Data embedding and extractiononly take place in expandable and exchangeable differences. Inessence, Tian’s algorithm relies on expandable differences toprovide spare space for data hiding and exchangeable differ-ences to guarantee blind data extraction. All difference valuesin the difference image are classified into three groups: expand-able, exchangeable, and nonexchangeable. Four disjoint sets arefurther defined to describe these three groups. For a payload,a number of expandable differences are selected from the ex-pandable sets. The corresponding location map of selected dif-ferences is then obtained. The compressed location map is savedalong with the payload for blind data extraction. Due to mul-tiple-layer embedding and inefficient compression of locationmap, Tian’s algorithm often has the problems of layer embed-ding capacity limit and low image quality.

During implementation, Tian’s sets-based difference selec-tion scheme is indirect and complicated. In particular, it is incon-venient for capacity control. To facilitate the selection processand capacity control, Thodi et al. [14] introduced a more in-tuitive scheme. They used a difference histogram to describethe statistical distribution of expandable differences. The his-togram is divided into two types of regions: the inner region

(to be embedded) that consists of the bins of selected expand-able differences and the outer regions (to be shifted) that con-tain the bins of unselected differences. By directly shifting theselection threshold along the difference axis, they could changethe size of inner and outer regions, and thus, assign a propernumber of expandable differences for the embedding of a pay-load. To ensure lossless recovery of the original image, thesetwo regions are kept not intersected with each other, even afterembedding. Thodi et al. used a histogram shifting operationto avoid region-overlapping. In order to avoid overflow/under-flow of embedded/expanded pixel values, this histogram onlyconsists of expandable differences. All possible overflow dif-ferences are expelled from the histogram in advance and theirlocations are recorded by an overflow location map. From nowon, we use overflow to denote either overflow or underflow forthe sake of simplicity. Such a map is saved along with the pay-load for the reversibility of the embedding process. The his-togram-based difference selection and shifting scheme and thegood compressibility of the overflow location map enable Thodiet al.’s method to be an efficient algorithm. There are five ver-sions of the algorithm in [14], where DE-HS-OM (DE with his-togram shifting and overflow map) and DE-HS-FB (DE withhistogram shifting and flag bits) methods are based on integerHaar wavelet transform domain, and the rest three are based onimage prediction errors. For large payloads, the prediction-error(PE)-based methods often have better performance; for smallpayloads, however, the integer Haar wavelet transform domain(DE-based) ones are more suitable. In general, these two typesof methods have different advantages. More detailed informa-tion can be found in [14]. Usually, DE-HS-OM and DE-HS-FBmethods have close performance. Like all other integer Haarwavelet transform domain methods, both of them suffer fromthe problems mentioned in Section I.

Basically, every method using this type of difference imagesfaces the same problem. The reason is multiple-layer embed-ding. For example, for a payload, the integer Haar wavelet trans-form domain method first searches for and uses small expand-able differences in one difference image. With the increase ofpayload size, it selects more and more large differences to obtainenough embeddable locations. Note that small or large differ-ences refer to differences with small or large magnitude in thispaper. When the payload size is close to 0.5 bpp, the method al-most uses up all expandable differences in the current differenceimage, including the largest possible differences. If the payloadsize continues to increase, this method can not satisfy the ca-pacity requirement in the single layer embedding but dependson multiple-layer embedding. However, only after an exhaustedsearch for expandable differences can not satisfy the payload,the method goes to the next layer embedding, which often takesplace in the difference image obtained in orthogonal transformdirection. Obviously, before beginning the next layer embed-ding, the image quality has been greatly degraded due to the useof large differences. One way to solve this problem is to assignthe payload to the two difference images obtained in orthogonaltransform directions, so that the differences under the same levelof magnitude in the two difference images can be used. It couldavoid the situation that there is few chance to use small differ-ences in the second difference image while all large expandable



Fig. 1. Algorithm diagram.

differences are used up in the first difference image. This is theinitial motivation of this work.

III. ALGORITHM

For simplicity of description, we use to represent theimage and use HH, HL, VH, and VL as its subscripts to denotehorizontal difference/high-frequency image, horizontal av-erage/low-frequency image, vertical difference/high-frequencyimage, and vertical average/low-frequency image, respectively.

We further define the following notations.

Payload.

Embedding capacity.

Auxiliary data package.

Header file.

2-D binary overflow location map.

Bitstream of compressed 2-D binary overflow locationmap.Selection threshold.

Zero point.

Embedded bitstream.

Function used for calculating the length of a bitstream.

We use a single capital letter H or V as the subscript to de-note the above symbols for the horizontal or vertical differenceimage.

The main phases of the proposed algorithm are exhibited inFig. 1. We first perform integer Haar wavelet transform in rowdirection to decompose the image into and . Then,the improved histogram-based difference selection and shiftingscheme is carried out in to embed the horizontal payload

. Similarly, in column direction, we have and .The improved histogram-based difference selection and shiftingscheme is performed in to embed the vertical payload .The division of the payload into and depends on thedynamical expandable difference search and selection mecha-nism, which is the focus of this work. It is worth mentioning thatFig. 1 only illustrates the situation that the embedding processbegins from the horizontal difference image. If the embeddingprocess begins from the vertical difference image, a similar di-agram can be obtained. Below we first describe our histogram-based difference selection and shifting scheme, and then, de-scribe the dynamical expandable difference search and selec-tion mechanism. Afterwards, we discuss data hiding. In the last



Fig. 2. Sample difference histogram.

subsection, we show how to perform data extraction and the re-covery of the original image.

A. Scheme for Selecting Expandable Differences

Our histogram-based difference selection and shiftingscheme is an improved version of the scheme in [14]. Webriefly review that scheme before introducing our improvedone.

1) Histogram-Based Difference Selection and ShiftingScheme: Thodi et al. [14] selected expandable differences/lo-cations based on the difference histogram. Since the differencevalues of the difference image approximately follow theLaplace distribution, Thodi et al. designed an outward his-togram shifting scheme to give priority to the bins with smalldifferences in the process of difference selection. We use Fig. 2to make a concise explanation.

In Fig. 2, the regions occupied by the bins with selected andunselected expandable differences are defined as the inner re-gion and the outer regions, respectively. Assume the selectionthreshold . The selected bins in the inner region have dif-ference values in the range . After DE embedding,the use of the embedding rule (3) makes the inner region ex-panded to become . Apparently, the expandedparts would overlap theoriginal outer regions. The original difference values in the over-lapped regions would be permanently lost if no action is takento avoid such an overlap. Therefore, before embedding, Thodi etal. shifted the left outer region and the right one outwards alongthe difference axis by at least , respectively.

2) Improved Histogram-Based Difference Selection andShifting Scheme: Although the above scheme is simple and ef-ficient, it still has some weakness. We improve it in two aspects.We first discuss how to use the histogram zero points, and then,we introduce a payload-dependent overflow location map.

Basically, histogram shifting implicitly corresponds to al-tering difference values. It brings about numerical distortionto the image. Therefore, we should decrease the histogramshifting distance as much as possible. One way to reducethe shifting distance is to use zero points. We use Figs. 2and 3 to explain this idea. Suppose in Fig. 2. Theinner region is and the corresponding outer regionsare and . After DE embedding, the inner

Fig. 3. Sample difference histogram with zero points in its tails.

region is expanded and becomes . According to theshifting strategy in [14], prior to embedding, the outer regionsshould be shifted outwards to become and ,respectively. However, the difference histogram derived from anatural image may be like what shown in Fig. 3. So the size ofshifted intervals/segments as well as the shifting distance canbe reduced by using zero points. A zero point is usually definedas with . In this work, however, we give onemore constraint. A zero point should not be located in the innerregion but in or . Theoretically,a difference value coming from an 8-bit image falls into therange though the real difference histogram is muchnarrower. However, only zero points within outer regions ofthe histogram contribute to our improved histogram shiftingscheme. A useful zero point should not be greater than 126or less than because the largest inner region occurs at

under the embedding rule (3).It can be observed in Fig. 3 that there are four zero points,

i.e., , , , and , within the tails of the histogramand two zero points, i.e., and 34, as the histogram ends.These zero points initially divide the histogram into severalsegments. Precisely, the right outer region has three segments:

, , and , and the left outer region alsohas three segments: , , and .Apparently, for some , we do not need to shift the whole outerregions. We use an example to explain the situation. When

, only is shifted to , and shiftedto . It is unnecessary to shift the other four segments.When , the right segments and are shiftedto and , respectively. At the same time, the leftsegments and are shifted toand , respectively. Still, two segments are left in-tact. When , , , become ,

, and , respectively; and meanwhile, ,, and become , ,

and , respectively. Obviously, only after ,that is, all of the zero points within the histogram are used up,the whole outer region would be shifted. Even so, four of theoriginal segments divided by these zero points do not shift asmuch as the original cases (without using zero points) do. Thisimprovement effectively reduces the alteration to the differencevalues in the outer regions.



We further investigate the use of zero points. Generally, theeffect of using zero points depends on two factors: image char-acteristics and payload size. If an image has many zero pointswithin its histogram, the outer regions are often divided into sep-arate segments. Using zero points would greatly reduce the al-teration to the histogram during histogram shifting. On the otherhand, if the payload size is small, the inner region is small andthe corresponding outer regions are large. There may be morezero points within the outer regions. Using zero points has astronger effect. However, as the payload size increases, the innerregion becomes larger. There are less zero points left in the outerregions. The effect of using zero points becomes weaker. Westill use the above example to explain this situation. Supposethe payload is large and the inner region has to befor satisfying this payload. The corresponding outer regions are

, and . There is only one zero point,i.e., 31, left in the outer regions. Apparently, the effect of usingzero points becomes much weaker. Since are nowlocated in the inner region, they are no longer zero points. Wetreat these points as common locations in the inner region anddetermine whether they are expandable or not using (4).

Now we discuss how to improve the overflow location map.As mentioned before, in order to ensure no overflow occur-ring during embedding and shifting, the difference histogram in[14] only consists of expandable differences. To match the his-togram, a payload-independent overflow location map is con-structed before using the histogram, which records all of pos-sible overflow locations in the difference image. Although thatpayload-independent overflow location map has efficient com-pression for common images, it often can not maintain goodcompression for other types of images, e.g., images with richtextures. In this work, we present an overflow location map thatcan suit different types of images. This overflow location mapis payload-dependent and varies with the change of inner andouter regions. It can achieve much higher compression ratio.Basically, our algorithm adopts a quite different difference his-togram. It consists of all differences of the difference image, in-cluding expandable, exchangeable and unexchangeable differ-ences, rather than only expandable differences in Thodi et al.’shistogram. The proposed overflow location map is used to matchour new difference histogram. To correctly construct the over-flow location map, we give the constraints on embedding andshifting operations below.

The construction of a 2-D binary overflow location mapbegins from initialization. is initialized with “0”. If a dif-ference value causes overflow, the corresponding location ischanged to “1”. describes the state of a difference in the innerregion or outer regions.

• For a difference in the inner region, we judge it using (4).If does not satisfy (4), its location in is indicated with“1”; otherwise, we do not alter .

• For a difference in the outer regions, we first calculatethe shifted difference value using , where

is the shifting distance correspondingto a segment divided by zero points. We then determinewhether causes overflow or underflow by (5). Note thatwe substitute for when using (5). If (5) is not satisfied,its location in is indicated with “1”; otherwise, we donot alter . A difference value that satisfies (5) is definedas a shiftable difference in this work. The reason of using

(5) is that the shifted difference values may be performedwith LSB replacement during data encoding. We will ex-plain the LSB replacement operation in Section III-C.

So far, we have described the improved histogram-baseddifference selection and shifting scheme. To sum up, thehistogram-based difference selection process is the processdetermining the selection threshold for a given payload. Thesearch process begins with . We increase by 1 eachtime until the right is found. The expansion of the innerregion occurs after the outward shifting of the outer regions.The zero points are used for decreasing the shifting distance ofthe outer regions. Since the matrix of our payload-dependentoverflow location map is often sparser than that in [14], it hashigher compression efficiency.

B. Dynamical Search for Expandable Differences inHorizontal and Vertical Difference Images

We carry out our difference selection scheme to search forexpandable locations in both horizontal and vertical differenceimages. To illustrate our two-direction search mechanism, wefurther give Fig. 4 focusing on the parts of interest in Fig. 1. Weuse Figs. 1 and 4 to explain each step of our dynamical searchmechanism.

• Step 1: For an image , we perform integer Haar wavelettransform in row direction and obtain the horizontal differ-ence image .

• Step 2: We then calculate the histogram of .• Step 3: We perform the improved histogram-based differ-

ence selection and shifting scheme. Suppose the payload as. The initial difference selection threshold is 0, but it in-

creases by 1 in each new search round. Suppose the currentselection threshold and the horizontal embedding capacityare and , respectively. In essence, is the numberof selected expandable differences under . So the re-quired capacity . Wewill explain why we embed and in the next sub-section. We first judge whether .This is the basic requirement. The set of selected expand-able differences supplies spare space for the payload onlyafter it has enough locations for the auxiliary data that con-sists of and . If this condition is not satisfied, wedirectly increase by 1 and repeat Step 3 again. Other-wise, we go to the next judgement. If , the hori-zontal capacity under is not large enough for . Onlypartial payload, i.e., , will beembedded in this direction. We need the additional embed-ding capacity and have to search in the vertical differenceimage. If , the horizontal capacity under islarge enough for . We do not need any more locations. Ineither of these two cases, we obtain the horizontally em-bedded difference image by performing data hiding,which we will further discuss in Section III-C. However,for the former case, is the preparation for Steps 4–8,whereas for the latter case, is the preparation for theend of the search process. The latter case happens when apayload is small. We use Fig. 5 to further explain the oper-ations in Step 3.

• Step 4: We obtain the horizontally embedded image bysynthesizing and the average image in Fig. 1.



Fig. 4. Dynamical expandable difference search and selection mechanism.

Fig. 5. Explanation of step 3.

• Step 5: We perform integer Haar wavelet transform onin column direction to obtain the vertical difference image

.• Step 6: We then calculate the difference histogram of .• Step 7: We perform the improved histogram-based differ-

ence selection and shifting scheme. The vertical payloadis the above required capacity , i.e., . We

use another inner search loop to test whether the verticalembedding capacity under , called , satisfies

, where . Theinner search loop begins with . In each pass, in-creases by 1. The search process continues until .Note that, no matter how we increase , the vertical selec-tion threshold should never be greater than the horizontalthreshold, i.e., . If the final is greater than0, the search round under current fails. We have toincrease by 1 and repeat the Steps 3–7 again. Once

, it implies that the vertical difference image canprovide enough embedding capacity for the rest of the pay-load. We then perform data hiding and obtain the verticallyembedded difference image . We use Fig. 6 to furtherexplain the operations in Step 7.

• Step 8: By performing inverse integer Haar wavelet trans-form, we synthesize the vertically embedded differenceimage and the average image in Fig. 1 to obtainthe output image .

Fig. 6. Explanation of step 7.

C. Data Hiding

As seen in Fig. 1, data hiding is carried out during the dynam-ical expandable difference search process. In practice, prior toeach search in the vertical difference image, we have to performdata hiding in the horizontal difference image to get ,from which we derive . Below we discuss the data hidingoperations. We first show how to construct the header file.

1) Construction of Header File: For blind data extraction andthe lossless recovery of the original image, we use the header file

to record all of embedding parameters for the reversibility ofthe embedding process. Since data hiding takes place in bothhorizontal and vertical directions, we need two header files. Weuse the horizontal header file to explain the constructionprocess. The vertical header file is constructed in a similarway.

where is a hexadecimal number that denotes the layer embed-ding information. and refer to horizontalembedding and vertical embedding in a single layer embedding



TABLE IRULES FOR ORIGINAL IMAGE RECOVERY. � AND � ARE THE MAXIMUM ZERO POINTS IN THE

POSITIVE AND NEGATIVE ZERO POINTS ARRAYS, RESPECTIVELY. � REFERS TO AN ELEMENT OF �

Fig. 7. Test images of �� . From left to right: Lena, F-16, Baboon, and Stream and bridge.

mode, respectively. and refer to hor-izontal embedding and vertical embedding in a two-layer em-bedding mode, respectively. and represent the arraysof zero points in the positive difference axis and the negativedifference axis, respectively. In fact, for the selection threshold

, the shifting of right outer region passes throughpositive zero points, and the shifting of left outer region passesthrough negative zero points. So we need these zeropoints in the process of data extraction. andrefer to the length information of the compressed overflow lo-cation map and the embedded payload, respectively.

2) Embedding for Blind Data Extraction: The data hidingprocess is identical in both horizontal and vertical differenceimages, so we give a description without indicating the spe-cific embedding direction. As stated in the last subsection, theauxiliary data , where denotes the concatena-tion operation and is the compressed obtained by usingJBIG-kit in [15]. is a necessity for blind data extraction. Con-sequently, the bitstream to be embedded is constructed as

. The number of selected expandabledifferences should be equal to , so that the bitstream bitsare embedded into the selected expandable difference values bitby bit using (3). However, at the decoder, we could not extractthe hidden data if we had taken such a manner of embedding.In fact, without the overflow location map and the embeddingparameters, we could not know the locations of the selected dif-ferences. It means that even we have determined the bitstreamas well as the expandable locations for hiding, we have to makesuch a hiding process easy for blind data extraction.

In [6], Tian implemented blind data extraction using the LSBsof changeable differences. Before embedding, the original LSBsof both unselected expandable differences and exchangeabledifferences are saved into a set. Then, the combined bitstreamof the compressed location map, the LSBs set and the payloadare embedded in sequence into the LSBs of expandable and ex-changeable differences. During blind data extraction, according

to the properties of exchangeable differences and expandabledifferences, the combined bitstream can be extracted from theLSBs of the exchangeable differences, so that the original imagecan be recovered losslessly. The reader is referred to [6] formore detailed information.

We borrow the idea from [6]. Data hiding begins from theupper left corner of the difference image in a manner of rasterscan. We save original LSBs of both shiftable differences andexchangeable differences in a set . Unlike [6], we only savethe LSBs of the first difference image pixels. In otherwords, we change the amount of difference image pixels as lessas possible in order to keep embedding distortion to the lowestdegree. We append the bitstream to . Apparently, bitsare embedded by DE, and bits are embedded by LSB re-placement. We implement data hiding in the following steps.

• Step 1: We scan the difference image in a raster scanmanner and perform data hiding from the upper leftcorner.

• Step 2: If the difference image pixel is in the inner regionand expandable, we hide one bit of the to-be-embeddedbitstream into using (3).

• Step 3: If is in the outer regions and shiftable, we shiftit based on its segment using our shifting scheme to obtain

. We then save the LSB of into in order. After that,we hide one bit of the to-be-embedded bitstream intousing LSB replacement.

• Step 4: If is neither expandable nor shiftable, but it isexchangeable, we first save the LSB of into . Afterthat, we hide one bit into using LSB replacement.

• Step 5: If may lead to overflow, we skip it and go to thenext pixel.

• Step 6: We repeat Steps 2–5 until we save LSBs into. After that, we only perform Step 2 and hide bits into

expandable differences. As soon as all bits of bitstreamare hidden, we hide the bits of immediately. The hidingoperation proceeds until we hide all bits of and .



Fig. 8. Embedding capacity versus image quality curves. We compare the proposed algorithm with Thodi et al.’s DE-HS-OM algorithm, Tian’s DE algorithm,and Kamstra et al.’s Extended DE algorithm at different embedding rates.

D. Data Extraction and Lossless Recovery

Data extraction and the lossless recovery of the original imageare implemented in two separate processes. The latter processneeds the result of the former. We first discuss data extrac-tion. According to [6], a changeable location remains change-able even after its LSB is replaced, whereas an expandable loca-tion may not be expandable after DE embedding, but it remainschangeable.

• Step 1: For a test difference image, we begin data extractionfrom the upper left corner of the image in a manner of rasterscan, just as the data hiding process does.

• Step 2: We extract the LSBs of difference image pixels toresume the embedding parameters. According to the struc-ture of , the LSBs of the first 16 difference image pixelscorrespond to . The following 8 LSBs correspond to theselection threshold . The next two LSBs se-quences correspond to the arrays of positive and negativezero points in turn. After that, we get the length informationof the overflow location map and the length infor-mation of the embedded payload from the following two24 LSBs sequences, respectively. So we have obtained thecomplete information bits of the header file .

• Step 3: According to the order of data hiding, the followingLSBs constitute the bitstream of compressed over-

flow location map . So we decompress using inverseJBIG compression to get back the original overflow lo-cation map . Thereafter, with the obtained and ,we can extract the rest of hidden bits from the differenceimage.

• Step 4: Due to , we are able to recalculate theposition where we stop performing LSB replacement in theprocess of data hiding. We keep the operation of extractingLSBs from the difference image pixels until we reach thatposition. After that position, we only extract the LSBs fromexpanded differences. After the first LSBs, the ex-tracted bits belong to two parts: the first bits are thepayload bits hidden in this difference image; the rest bitsare the elements of .

After data extraction is finished, we can begin the image re-covery process. From the upper left corner, we scan the differ-ence image again and resume each difference image pixel. Therecovery operations depend on , , and . We list the rulesfor recovery in the Table I. The recovery process proceeds untilall the difference image pixels are scanned.



IV. EXPERIMENT AND DISCUSSION

We have tested the proposed algorithm on different types ofimages. In this work, we give the experimental results of fourtest images that are shown in Fig. 7. We focus on comparingour algorithm with Thodi et al.’s DE-HS-OM algorithm in [14],which is one of the latest DE-based algorithms in literature. Wealso simply compare our algorithm with Tian’s original DE al-gorithm in [6], and Kamstra et al.’s Extended DE algorithm in[9]. Extended DE algorithm is an improved version of Tian’sDE algorithm. Since Extended DE algorithm uses the charac-teristics of the low-pass image to predict expandable locations,it often has good performance under small payloads. More de-tailed information can be found in [9]. For simplicity, we call ouralgorithm as DE-TED in Fig. 8 and Table II, where Fig. 8 givesthe embedding capacity versus image quality curves of four al-gorithms and Table II gives the detailed results of our algo-rithm. For better comparison, Table II also gives the detailed re-sults of DE-HS-OM algorithm. The payloads 14 000 bits, 27 000bits,…, 210 000 bits, 237000 bits approximately correspond to0.05 bpp, 0.1 bpp, 0.2 bpp,…, 0.8 bpp, 0.9 bpp, in sequence.

For common images like Lena, when the embedding rate isless than 0.15 bpp, our algorithm and DE-HS-OM algorithmproduce close PSNR values. The main reason is that our al-gorithm only uses one embedding direction under such smallpayloads, so does DE-HS-OM algorithm, as shown in Table II.From 0.2 bpp to 0.4 bpp, however, there are some differencebetween our algorithm and DE-HS-OM algorithm in imagequality, and this difference tends to be larger with the increaseof payload size. Table II exhibits that our algorithm takes twoembedding directions at these embedding rates, but DE-HS-OMalgorithm still embeds in one layer; moreover, under the samepayload, our algorithm uses smaller selection thresholds thanDE-HS-OM algorithm does. It implies that our algorithm usessmaller differences for embedding. Therefore, our algorithmproduces higher PSNR values. Since the difference betweenour selection thresholds and DE-HS-OM algorithm’s counter-parts is not large under these payloads, the difference betweenour algorithm’s PSNR values and DE-HS-OM algorithm’scounterparts is not large. The biggest difference between ouralgorithm and DE-HS-OM algorithm in PSNR values occursat 0.5 bpp, as shown in Fig. 8(a). The PSNR value producedby the former is about 5 dB higher than that produced by thelatter. This phenomenon can be easily explained by Table II.At 0.5 bpp, our algorithm has for vertical embeddingdirection and for horizontal embedding direction,whereas DE-HS-OM algorithm has for the first layerembedding and for the second layer embedding. Theuse of large differences in its first layer embedding certainlydamages the performance of DE-HS-OM algorithm. From0.5 bpp to 0.9 bpp, our algorithm still performs better thanDE-HS-OM algorithm. But with the increase of payload size,our algorithm uses larger selection thresholds, and thus, ouradvantage over DE-HS-OM algorithm becomes weaker. Whenthe payload size approaches the maximum capacity, the dif-ference between our algorithm and DE-HS-OM algorithm is

very small. With regard to DE and Extended DE algorithms,the latter performs better than the former when the embeddingrate is less than 0.4 bpp, but the former performs a little betterthan the latter from 0.5 bpp. Our algorithm outperforms bothof them, especially, around 0.5 bpp. It is worth mentioning thatthe good performance of our algorithm for Lena is mainly dueto the dynamical expandable difference search and selectionmechanism. Since Lena has a sharp and narrow differencehistogram with few zero points in its tails, the use of zero pointsand the payload-dependent overflow location map has a weakcontribution to the algorithm performance.

For flat images like , the situation is similar to thatfor Lena, but the advantage of our algorithm is more obvious.This is because a flat image is more sensitive to the alterationof image values. It can be seen in Fig. 8(b) that the embeddingcapacity versus image quality curves of DE-HS-OM, DE, andExtended DE algorithms have the sharpest dip around 0.5 bppamong the four test images. Our embedding capacity versusimage quality curve is very smooth. At 0.5 bpp, the PSNR valueof our algorithm is about 11 dB higher than any of the otherthree. Our algorithm respectively uses andfor vertical and horizontal embedding directions, and producesa PSNR value of 43 dB; meanwhile, DE-HS-OM algorithm re-spectively uses 126 and 0 for its first- and secondlayer embed-ding, and produces a PSNR value of 32 dB. DE and ExtendedDE algorithms also face layer embedding capacity limitation,so they produce similar PSNR values as DE-HS-OM algorithmdoes. Note that, although the smoothness of the low-pass imageof enables Extended DE algorithm to have a better per-formance than our algorithm under small payloads, its perfor-mance worsens quickly as the embedding rate is greater than0.3 bpp.

For texture images like Baboon, our algorithm performsbetter than either DE-HS-OM or DE algorithm at all embed-ding rates. DE algorithm has the worst embedding capacityversus image quality performance among the four test images.The biggest difference between our algorithm and these twoalgorithms in PSNR values still occurs at 0.5 bpp. The good per-formance of our algorithm mainly comes from two sources: thedynamical expandable difference search and selection mech-anism and the high compressibility of our payload-dependentoverflow location map. According to Table II, from 0.05 bpp to0.8 bpp, our algorithm gradually increases the selection thresh-olds, for example, ; meanwhile,DE-HS-OM algorithm increases the selection thresholds veryquickly and uses the selection thresholds 2, 4, 9, 19, 41, 126,126, 126, 126 in sequence for its first layer embedding. It isclear that DE-HS-OM algorithm uses much larger differencesthan our algorithm. The use of small differences certainlybenefits our image quality. In addition, our payload-dependentoverflow location map also shows its advantages for Baboonwhose difference histogram is flat and wide. It can be seen thatthe size of our compressed location map is small and varies withpayloads; moreover, the increase in size is small even when thepayload becomes very large. Specifically, when the payload is14 000 bits, the size of our compressed overflow location map



TABLE IIDETAILED RESULTS. WE COMPARE OUR ALGORITHM WITH THODI et al.’S DE-HS-OM ALGORITHM. � , T AND� REFER TO THE PAYLOAD, THE SELECTION

THRESHOLD AND THE COMPRESSED OVERFLOW LOCATION MAP, RESPECTIVELY. THE UNIT OF � AND� IS ��. � AND � REFER TO THE VERTICAL AND

HORIZONTAL EMBEDDING DIRECTIONS, RESPECTIVELY. “-” REFERS TO UNAVAILABLE. THE VERTICAL DIRECTION IS USED AS THE FIRST EMBEDDING DIRECTION

in the vertical difference image is 824 bits. When the payloadbecomes 210000 bits, the sizes of our compressed overflowlocation maps in the vertical and horizontal difference imagesare 1552 bits and 4208 bits, respectively. In contrast, theircounterparts of DE-HS-OM algorithm are payload-indepen-dent, i.e., 6960 bits in the vertical difference image and 25 264bits in the horizontal difference image. Apparently, when thepayload is only 14 000 bits, saving a compressed overflow

location map of 6960 bits is an important factor that affectsimage quality. Even when the payload is 210000 bits, saving acompressed overflow location map of 25 264 bits is a burden.Basically, such a low efficient compression of the second over-flow location map is due to the improper embedding in the firstdifference image. On the other hand, our payload-dependentoverflow location map is compression-efficient. It can save alot of spare space for this type of images. As for Extended DE



TABLE IIIIMAGE QUALITY DIFFERENCE CAUSED BY EMBEDDING DIRECTIONS. THE RESULTS ARE OBTAINED BY PERFORMING TIAN’S DE ALGORITHM ON LENA. PSNR1

AND PSNR2 DENOTE THE PSNR VALUES OBTAINED FROM HORIZONTAL-VERTICAL EMBEDDING AND VERTICAL-HORIZONTAL EMBEDDING, RESPECTIVELY.

algorithm, its performance is better than either DE-HS-OM orDE algorithm’s performance, but still below our algorithm’sperformance from 0.05 bpp.

For images with some textures and uneven brightness likeStream and bridge, the advantage of our algorithm is more clear.Fig. 8(d) shows that, when the embedding rate is less than 0.1bpp, the PSNR value of our algorithm is at least 10 dB higherthan that of DE-HS-OM algorithm or Extended DE algorithm,and 20 dB higher than that of DE algorithm. In addition, be-tween 0.1 bpp–0.5 bpp, the PSNR value of our algorithm isat least 5 dB higher than any of the other three. We investi-gate our algorithm performance from three aspects. In Table II,under the same payload, our algorithm uses much smaller se-lection thresholds than DE-HS-OM algorithm does; besides, thesize of our compressed overflow location map is much smallerthan its counterpart of DE-HS-OM algorithm. These two advan-tages demonstrate that both the dynamical expandable differ-ence search and selection mechanism and our payload-depen-dent overflow location map contribute to our algorithm perfor-mance. However, another major contribution comes from usingzero points. Unlike Lena or that has a sharp and narrowdifference histogram, or Baboon that has a flat but continuousdifference histogram, Stream and bridge has a difference his-togram that is not sharp but ragged and with many zero pointsclose to the origin. Specifically, the closest zero points are 1 onthe positive difference axis and on the negative differenceaxis. Those zero points divide the outer regions into many smallsegments, which facilitate the use of zero points. For a smallpayload, our algorithm only shifts a small portion of differences,so that the distortion is very small. That is why our algorithmperforms even better than Extended DE algorithm. Usually, Ex-tended DE algorithm has good performance at small embeddingrates, as seen in Fig. 8(a)–(c). The effect of using zero points be-comes weak as the payload increases.

From the results of these four test images, we conclude thatour algorithm not only yields higher image quality and over-comes the abrupt change in the algorithm performance near thelayer embedding rate limit, but also has better resilience to dif-ferent types of images. Table II clearly indicates that our algo-rithm uses the dynamical expandable difference search and se-lection mechanism to keep and not far from each other,and the magnitude difference between them is at most 2. In con-trast, the DE-HS-OM algorithm goes to the second differenceimage unless the first difference image can not bear any morepayload. Thus, the largest magnitude difference between its twoselection thresholds is 126. Table II also shows that our algo-rithm does not evenly assigns the payload to the two embedding

directions, but make efforts on evenly selecting expandable dif-ferences under the same threshold in two embedding directions.We have to point out that the results in Fig. 8 and Table II areobtained by using the vertical direction as the first embeddingdirection. Although there is no much difference in experimentalresults for our algorithm between using the vertical directionas the first embedding direction and using the horizontal direc-tion as the first embedding direction, the difference does existfor the other three algorithms. In other words, quite different re-sults may appear in Fig. 8 and Table II for these three algorithmsif using the horizontal direction as the first embedding direc-tion. We briefly discuss the problem using Table III. To obtainPSNR1, we use the horizontal direction as the first embeddingdirection. To obtain PSNR2, we use the vertical direction as thefirst embedding direction. The difference between PSNR1 andPSNR2 is clearly revealed by Table III. Under the same em-bedding rate, the largest PSNR difference is about 2 dB. Thisphenomenon can be theoretically explained. In our algorithm,we always select differences under the same threshold in twodifference images, so the embedding in the first direction wouldnot severely affect the correlation among image pixels. How-ever, the other three algorithms use up all possible expandabledifferences in the first difference image before using expandabledifferences in the second difference image. Since statistical cor-relation among pixels in the vertical difference image is oftendifferent from that in the horizontal difference image, the choiceof first embedding direction often leads to different results. Thisdisadvantage is certainly not good for real-world applications.

Prediction-error-based embedding is also a popular reversibledata hiding technique in literature. Here we grossly compareour algorithm with P2 (PE expansion with histogram shiftingand overflow map) method, which is the best PE version ofThodi et al.’s algorithm in [14]. We directly use the data pro-vided by Thodi et al.. For Lena, from 0.1 bpp to 0.9 bpp, P2approximately produces PSNR values 47 dB, 46 dB, 44 dB,42 dB, 40 dB, 38 dB, 37 dB, 36 dB, and 34 dB in sequence;and meanwhile, our algorithm produces PSNR values 52 dB,49 dB, 46 dB, 43 dB, 41dB, 40 dB, 38 dB, 36 dB, and 34dB in sequence. It can be observed that, the PSNR values ofour algorithm are much higher than those of P2 when the em-bedding rate is less than 0.3 bpp. From 0.4 bpp, our algorithmis still better than or as good as P2. According to the data in[14], the similar conclusion can be obtained for both Baboonand

. Actually, just as stated in [14], DE methods are betterthan PE-based methods at low embedding rates. Thodi et al.also stated that PE-based methods perform better at high em-bedding rates. However, due to our improvement, the proposed



DE-based method is still better than or as good as PE-basedmethods at medium and high embedding rates.

We briefly discuss the computational complexity of our algo-rithm before ending this section. Our simulation is carried out ona Pentium-4 3.44 GHz PC with 1G RAM. Our algorithm is im-plemented in Visual C 6.0. For Lena, from 0.1 bpp to 0.9 bpp,the embedding process takes about 0.250 s, 0.500 s, 0.594 s,0.750 s, 1.000 s, 1.312 s, 1.718 s, 2.984 s, and 6.860 s, respec-tively. The corresponding detection process takes about 0.141 s,0.188 s, 0.219 s, 0.235 s,0.239 s, 0.242 s, 0.245 s, 0.247 s, and0.250 s, respectively. It can be seen that the computational com-plexity of our algorithm is acceptable for general applications.When the embedding rate is close to the maximum capacity, theconsuming time will be longer. However, the image quality ob-tained at such high embedding rates is often not used for com-mercial purposes.

V. CONCLUSION

DE embedding algorithms available in literature often use upall expandable differences in the current difference image be-fore performing data embedding in the next difference image.Such a scheme may cause obvious distortion in the outputimage. Moreover, the algorithm can not perform smoothlynear the layer embedding capacity limit. In this paper, we haveproposed a new embedding algorithm to overcome these prob-lems. We introduce a dynamical expandable difference searchand selection mechanism to balance the use of differences intwo embedding directions. This mechanism effectively avoidssevere embedding distortion resulting from the use of largedifferences in the previous difference image. We also exploitzero points and a payload-dependent overflow location map toimprove the histogram-based difference selection and shiftingscheme. Using zero points can decrease the histogram shiftingdistance and using the payload-dependent overflow locationmap can increase compression efficiency of the overflow loca-tion map.

The proposed algorithm does not have the original layer em-bedding capacity limit. It can perform smoothly at different em-bedding rates. The experimental results have proven that our al-gorithm often outperforms Thodi et al.’s newly published algo-rithm, Tian’s original DE algorithm, and Kamstra et al.’s Ex-tended DE algorithm for different types of images. The advan-tage of our algorithm is more prominent around the embeddingrate of 0.5 bpp.

ACKNOWLEDGMENT

The authors would like to thank the Associate Editor, Prof. I.Ahmad, and the anonymous reviewers for their valuable com-ments that have helped us improve the quality and clarity of thepaper.

REFERENCES

[1] C. W. Honsinger, P. Jones, M. Rabbani, and J. C. Stoffel, “Lossless Re-covery of an Original Image Containing Embedded Data,” U.S. Patent6 278 791, 2001.

[2] B. Macq, “Lossless multiresolution transform for image authenticatingwatermarking,” in Proc. Eur. Signal Process. Conf., Sep. 2000, pp.533–536.

[3] W. N. Lie, C. I. Lin, D. C. Tsai, and G. S. Lin, “Error resilientcoding based on reversible data embedding technique for H.264/AVCvideo,” in Proc. IEEE Int. Conf. Multimedia and Expo, Jul. 2005, pp.1174–1177.

[4] R. Li, O. C. Au, C. Yuk, S.-K. Yip, and T.-W. Cha, “Enhanced imagetrans-coding using reversible data hiding,” in Proc. IEEE Int. Symp.Circuits Syst., May 2007, pp. 1273–1276.

[5] J. Fridrich, M. Goljan, and R. Du, “Lossless data embedding—newparadigm in digital watermarking,” EURASIP J. Appl. Signal Process.,Special Issue on Emerging Applications of Multimedia Data Hiding,vol. 2002, pp. 185–196, Feb. 2002.

[6] J. Tian, “Reversible data embedding using a difference expansion,”IEEE Trans. Circuits Syst. Video Technol., vol. 13, pp. 890–896, Aug.2003.

[7] A. M. Alattar, “Reversible watermark using the difference expansionof a generalized integer transform,” IEEE Trans. Image Process., vol.13, no. 8, pp. 1147–1156, Aug. 2004.

[8] M. U. Celik, G. Sharma, A. M. Tekalp, and E. Saber, “Lossless gener-alized-LSB data embedding,” IEEE Trans. Image Process., vol. 12, no.2, pp. 157–160, Feb. 2005.

[9] L. Kamstra and H. J. A. M. Heijmans, “Reversible data embeddinginto images using wavelet techniques and sorting,” IEEE Trans. ImageProcess., vol. 14, no. 12, pp. 2082–2090, Dec. 2005.

[10] Z. Ni, Y. Q. Shi, N. Ansari, and W. Su, “Reversible data hiding,” IEEETrans. Circuits Syst. Video Technol., vol. 16, pp. 354–362, Mar. 2006.

[11] Y. Hu and B. Jeon, “Reversible visible watermarking and lossless re-covery of original images,” IEEE Trans. Circuits Syst. Video Technol.,vol. 16, pp. 1423–1429, Nov. 2006.

[12] G. Xuan, Q. Yao, C. Yang, J. Gao, P. Chai, Y. Q. Shi, and Z. Ni,“Lossless data hiding using histogram shifting method based on in-teger wavelets,” in Int. Workshop Digital Watermarking, Nov. 2006,vol. 4283, pp. 323–332.

[13] D. Coltuc and J.-M. Chassery, “Very fast watermarking by reversiblecontrast mapping,” IEEE Signal Process. Lett., vol. 14, no. 4, pp.255–258, Apr. 2007.

[14] D. M. Thodi and J. J. Rodriguez, “Expansion embedding techniquesfor reversible watermarking,” IEEE Trans. Image Process., vol. 16, no.3, pp. 721–730, Mar. 2007.

[15] [Online]. Available: http://www.cl.cam.ac.uk/mgk25/jbigkit/

Yongjian Hu (M’05) received the Ph.D. degree fromSouth China University of Technology, Guangzhou,China, in 2002.

He is a Professor of the College of AutomationScience and Engineering, South China Universityof Technology. During 2000 to 2004, he visited theDepartment of Computer Science, City Universityof Hong Kong four times as a Research Assistant,Senior Research Associate, and Research Fellow,respectively. From 2005 to 2006, he worked as aResearch Professor in the School of Information and

Communication Engineering, Sung Kyun Kwan University, Korea. Since 2006,he has been visiting the Department of Computer Science, Korea AdvancedInstitute of Science and Technology (KAIST), Daejeon, Korea, as a VisitingProfessor. His research interests include information security, digital forensics,and image processing.

Heung-Kyu Lee received the B.S. degree in Elec-tronics Engineering from the Seoul National Univer-sity, Seoul, Korea, in 1978, and M.S. and Ph. D. De-grees in computer science from the Korea AdvancedInstitute of Science and Technology (KAIST), Dae-jeon, in 1981, and 1984, respectively.

From 1986, he is a Professor of the Department ofComputer Science, KAIST. He is an author/coauthorof more than 100 international journal and confer-ence papers. He is a reviewer for many internationaljournals, including Journal of Electronic Imaging,

Real-Time Imaging, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR

VIDEO TECHNOLOGY, etc. He was also a program chairman of many in-ternational conferences, including the International Workshop on DigitalWatermarking (IWDW) in 2004, IEEE International conference on Real-TimeSystems and Applications, etc. Since 2006, he has been a general director ofCenter of Fusion Technology for Security (CFTS). His major interests areinformation hiding and multimedia forensics.



Kaiying Chen received the B.E. degree from Xi’anUniversity of Science and Technology, Xi’an, China,in 2005. She is currently pursuing the M.S. degreein pattern recognition and intelligent systems in theCollege of Automation Science and Engineering,South China University of Technology, Guangzhou.Her research interests include digital image pro-cessing, data hiding and pattern recognition.

Jianwei Li received the B.E. degree from Xi’an Uni-versity of Technology, Xi’an, China, in 2005. He iscurrently pursuing his M.S. degree in pattern recog-nition and intelligent systems in the College of Au-tomation Science and Engineering, South China Uni-versity of Technology, Guangzhou. His research in-terests include digital image processing, data hidingand pattern recognition.


Date post:	21-Apr-2018
Category:	Documents
Upload:	dobao
View:	220 times
Download:	3 times

1500 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. …hklee.kaist.ac.kr/publications/IEEE Trans on...

Documents