1

Data-dependent Hashing Based on p-StableDistribution

Xiao Bai, Haichuan Yang, Jun Zhou, Peng Ren and Jian Cheng

Abstract—The p-stable distribution is traditionally used fordata-independent hashing. In this paper, we describe how toperform data-dependent hashing based on p-stable distribution.We commence by formulating the Euclidean distance preservingproperty in terms of variance estimation. Based on this property,we develop a projection method which maps the original datato arbitrary dimensional vectors. Each projection vector is alinear combination of multiple random vectors subject to p-stabledistribution, in which the weights for the linear combinati on arelearned based on the training data. An orthogonal matrix isthen learned data-dependently for minimizing the thresholdingerror in quantization. Combining the projection method andthe orthogonal matrix, we develop an unsupervised hashingscheme which preserves the Euclidean distance. Compared withdata-independent hashing methods, our method takes the datadistribution into consideration and gives more accurate hashingresults with compact hash codes. Different from many data-dependent hashing methods, our method accommodates multiplehash tables and is not restricted by the number of hash functions.To extend our method to a supervised scenario, we incorporate asupervised label propagation scheme into the proposed projectionmethod. This results in a supervised hashing scheme whichpreserves semantic similarity of data. Experimental results showthat our methods have outperformed several state-of-the-arthashing approaches in both effectiveness and efficiency.

I. I NTRODUCTION

The volume of image data has been increasing dramaticallyevery year. The big data era has created great challenges tomany tasks such as content-based image retrieval (CBIR). Onetypical example is the nearest neighbor (NN) search, whichfinds the nearest sample for a query represented as a vectorizeddescriptor inRd. It requires a distance metric be defined tomeasure the similarity between image descriptors, and theEuclidean distance is one of the most widely used metrics.In this scenario, the query time has linear dependence onthe data size, which is impractical for large scale database.For data with relatively low dimensionality, the problem canbe solved using tree based methods such as binary searchtree [1]. However, the dimensionality of most popular imagedescriptors, for example those constructed by the Bag-of-Words [2] or GIST [3], is too large. It degrades the efficiencyof these methods to that of exhaustive search [4].

X. Bai and H. Yang are with School of Computer Science andEngineering, Beihang University, Beijing 100191, China. (e-mail: baix-iao.buaa@googlemail.com.)

J. Zhou is with School of Information and Communication Technology,Griffith University, Nathan, QLD 4111, Australia.

P. Ren is with College of Information and Control Engineering, ChinaUniversity of Petroleum, Qingdao 257061, China.

J. Cheng is with National Lab of Pattern Recognition, Institute of Automa-tion, Chinese Academy of Sciences, Beijing 100190, China.

Approximate nearest neighbor (ANN) techniques have beenstudied to break the bottleneck of NN search. Its key idea is tofind an approximate nearest neighbor rather than the exact one.Locality-sensitive hashing (LSH) has been introduced for thispurpose [5] and has attracted lots of attention. Its objective is tomap the original vectorv ∈ R

d to a binary stringy ∈ {0, 1}r

such that neighboring samples in the original feature spacehave similar binary codes in the Hamming space. However,simple feature similarity such as that based on Euclideandistance in the original feature space usually cannot fullycapture the semantic similarity,i.e., the real affinity betweenthe contents of objects. For example, in CBIR applications,ifthe images are represented as GIST descriptors, the Euclideanmetric may result in some false positive instances for a givenquery. One possible solution for this problem is to introducingsupervised learning based strategies into hashing, which haveled to significant improvement of the CBIR performance.Hashing methods which only aim at preserving feature sim-ilarity are called unsupervised hashing, and those based onsupervised learning strategy are called supervised hashing.

Alternatively, Hashing based techniques can be classi-fied into two categories, data-dependent hashing or data-independent hashing, depending on whether or not they em-ploy a training set to learn the hash function. Data-independenthashing does not require training data. A typical exampleis the method presented in [6], which uses data-independentmathematical properties to guarantee that the probabilityofcollision between hash codes reflects the Euclidean distanceof samples. The performance of data-independent methods isrobust to the data variations because the hash functions areestablished subject to specific rules without the training pro-cess. The randomness property enables the data-independentmethods to generate arbitrary number of hash functions. Soone can construct multiple hash tables to boost the recallrate. However, such methods suffer from the high demand onthe dimensionality of binary representation,i.e., the lengthof codesr has to be very large in order to reduce the falsepositive rate. This increases the storage costs and degrades thequery efficiency.

Data-dependent hashing methods, on the contrary, aim atlearning hash functions from a training set. A common objec-tive is to explicitly make the similarity measured in the originalfeature space be preserved in the Hamming space [7], [8], [9],[10]. Some methods, such as kernelized locality sensitive hash-ing (KLSH) [11], do not have an explicit objective function butstill require a training set. Compared with data-independentcounterpart, data-dependent hashing methods allow compactcoding, which is very feasible in practice. A typical kind

2

...

Random

project

Learning

Data-independent Data-dependent

Orthogonal

transformation

...

MLSH-ITQ

0.3

-0.6

0.7

-1.21.2

-0.4

(c dims)

r times

n images

Assign quasi

hashbit

MLSH-SLP

... r times

... r times

1

-1

(r bits)nr-bit binarycodes

... r times

Threshold by

probabilityPropagation

Threshold by sign(·)

r times

0.8

1.1

(c dims)

Fig. 1. The proposed method on extending p-stable distribution theory to data-dependent hashing.

of data-dependent method is the supervised hashing whichnot only considers data distribution, but also incorporatesprior information such as class labels for supervised learning.The disadvantages of data-dependent methods is that theirperformance may be too dependent on the training set andthey usually have limited amount of hash functions.

We can see that both data-independent and data-dependentsolutions have their pros and cons. An intuitive idea toovercome their shortcomings is developing an integrated strat-egy which combines both data-dependent hashing and data-independent hashing, and makes them complementary to eachother. To achieve this goal, we propose a hashing methodbased on p-stable distribution. The p-stable distribution[12]is traditionally used in data-independent hashing methods[6].It has special mathematical properties that guarantee thedistance underlp norm to be recovered by the projections onspecific random vectors. In our work, we extend the p-stabledistribution to the data-dependent setting.

An overview of the proposed method is illustrated in Fig-ure 1. Firstly, we project one original feature vector throughmultiple random vectors, and learn a single projection vectorfor approximating the multiple random vectors according tothe data distribution. The same procedure is repeated forrtimes, and givesr projection vectors. This is different fromLSH, which directly uses a single random vector as eachprojection vector, and we thus refer to our method as multiplelocality sensitive hashing (MLSH). Based on MLSH, we thenapply an orthogonal transformation [13] to the obtained projec-tion vectors for preserving the Euclidean distance with binarycodes. Conveniently, we refer to this process as MLSH-ITQ(MLSH with iterative quantization). Furthermore, we use theprojection result of MLSH to assign quasi hash bits for sometraining samples and perform a label propagation [14] likeprocess with respect to the semantic similarity to generatehashbits for the rest. We refer to this supervised hashing methodas MLSH-SLP (MLSH with supervised label propagation).

In [15], we introduced the p-stable distribution theory intothe data-dependent hashing. This method consists of twostages. In the first stage, Gaussian random vector is directly

used to assign initial binary labels for a part of data. In thesecond stage, the labels of the rest data are induced accordingto the unsupervised similarity. In this paper, the proposedMLSH method follows a similar two-stage framework, butwith completely different strategies in both stages. In thefirst stage, it uses the refined projection vector based ondeeper analysis of the p-stable property. In the second stage,MLSH incorporates different ways based on two differentscenarios. For unsupervised scenario, iterative quantizationis incorporated to refine the hash functions for retrievingEuclidean neighbors. For supervised scenario, the supervisedlabel propagation procedure is used to learn the hash functionsfor retrieving semantically similar instances.

The contributions of this paper are summarized as follows.Firstly, based on p-stable distribution theory, we show howtoview the Euclidean distance preserving problem as estimatingthe variance of a p-stable distribution. This observation leadsto a novel projection method which maps the samples in theoriginal feature space to arbitrary dimensional real-valued vec-tors. For each dimension, rather than directly using one singlerandom vector, we generate its projection vector based onapproximating the multiple random vectors for recovering theEuclidean distance within the dataset. Secondly, based on thismapping, we show how the iterative quantization method [13]can be used for minimizing the loss of thresholding. This leadsto the development of the unsupervised hashing MLSH-ITQ.Finally, we construct an objective function which is similarto [7] but characterizes semantic similarity, and compute itsapproximate solution by combining the proposed projectionmethod with a coordinate descent algorithm. This results inanovel supervised hashing scheme for the purpose of preservingthe semantic similarity, which to a certain extent eliminates theinconsistency of feature similarity and semantic similarity inhashing.

In the rest of the paper, a review of relevant hashing methodsis given in Section II. The proposed unsupervised hashingis described in Section III, followed by the introduction ofa novel supervised hashing in Section IV. We present theexperimental results in Section V, then draw conclusions and

3

discuss the future work in Section VI.

II. RELATED WORK

Compared against its data dependent counterpart, data-independent methods are usually considered to be more adap-tive to heterogeneous data distributions, but with the decreaseof efficiency in practice [7]. Locality sensitive hashing basedon p-stable distribution (LSH) [6] is one of the most represen-tative methods in the data independent hashing category. Basedon the p-stable distribution, hash functions can be generateddirectly without any training data, and the mathematical prop-erties of p-stable distribution [16] guarantee that vectors closeto each other in the original feature space have high probabilityto generate the same output by each hash function. Each hashfunction is a random linear projection and is independentto each other because of the randomness of the projectionvectors. Some other data-independent hashing schemes havebeen proposed besides LSH. For example, in [17], a data-independent hashing scheme has been reported, which utilizesrandom Fourier features to make the Hamming distance berelated to the shift-invariant kernel (e.g., Gaussian kernel)between the vectors. Recently, a kernelized locality sensitivehashing (KLSH) [11] has been proposed. It constructs randomproject vectors by using a weighted sum of data instances ina training set to approximate the Gaussian random hyperplanein a highly implicit kernel space.

In many applications, the data distribution is not verycomplex and can be well learned from a training set. Inthis scenario, data-dependent approaches become very ap-pealing. A representative data-dependent hashing scheme isspectral hashing (SH) [7]. It transforms the problem of find-ing similarity preserving code for a given dataset to a NP-hard graph partitioning problem that is similar to Laplacianeigenmaps [18]. SH relaxes this problem and solve it bya spectral method [7], [18]. For novel data point, SH usesthe Laplace-Beltrami eigenfunctions to obtain binary codesunder the hypothesis that the data is uniformly distributed. Toaddress the problem when data do not meet this hypothesis,anchor graph hashing (AGH) [9] has been proposed. AGHuses an anchor graph to obtain a low-rank adjacency matrixwhich is computationally feasible to approximate the similaritymatrix and then processes it in constant time based on theNystrom method [19]. Zhanget al. proposed a self-taughthashing [20] method that firstly performs Laplacian eigenmapsand then thresholds eigen-vectors to get binary code for thetraining set. After that, it trains an SVM classifier as the hashfunction for each bit. Recently, more extensions of the abovemethods have been developed. For instance, multidimensionalspectral hashing [21] is guaranteed to maintain the affinitieswhen the number of bits increases. Liet al. extended thespectral hashing with semantically consistent graph in [22],which incorporates prior information into SH in a supervisedmanner. Furthermore, Shenet al. [23] have developed a groupof hashing techniques based on a wide variety of manifoldlearning approaches such as Laplacian eigenmaps.

Dimensionality reduction methods have been widely ap-plied into hashing problems. Several data-dependent hashing

methods have been developed based on Principal Compo-nent Analysis (PCA) [24], including PCA-Direct [13] whichdirectly thresholds the results after performing PCA, PCA-RR [25] which applies a random orthogonal transformationbefore thresholding, PCA-ITQ [13] which refines an orthogo-nal transformation to reduce quantization error, and IsotropicHashing [26] which learns orthogonal transformation thatmakes projected dimensions have equal variance. In [13],Gonget al. also presented a supervised hashing method CCA-ITQ based on Canonical Correlation Analysis (CCA) and thesame iterative quantization method. LDAHash [27] introducesLinear Discriminant Analysis (LDA) [28] into hashing forlocal descriptors matching. Binary Reconstructive Embedding(BRE) [8] and Minimal Loss Hashing (MLH) [10] optimizeobjective functions directly with respect to the binary code.BRE aims to reconstruct the Euclidean distance in the Ham-ming space, and MLH has a hinge-like loss function.

Various learning settings have been explored in data de-pendent hashing. Semi-supervised hashing (SSH) [29] hasbeen introduced to search semantic similar instances whenonly part of the data are labelled. It minimizes the empir-ical error over the labeled data, and applies an informationtheoretic regularizer over both labeled and unlabeled data.Projection learning hashing method [30] has been proposedin a similar form as SSH, containing a semi-supervisedmethod and an unsupervised method. Beside SSH, weakly-supervised hashing [31] and kernel-based supervised hashing(KSH) [32] are two other supervised hashing schemes thathave kernel based hash functions. Kuliset al. have extendedLSH functions to a learned metric [33], which can also beconsidered as a supervised method. Beside these methods,several other hashing methods have been proposed to addressdifferent aspects of the modelling and computation, includingsemantic hashing [34], random maximum margin hashing [35],Manhattan hashing [36], dual-bit quantization hashing [37],spherical hashing [38] and k-means hashing [39].

III. U NSUPERVISEDHASHING FORPRESERVING

EUCLIDEAN DISTANCE

In this section, we present our unsupervised hashing schemeMLSH-ITQ based on p-stable distribution. As illustrated inFigure 1, there are two major parts within our scheme,with one being data-independent and the other being data-dependent. The core idea is to use multiple random vectors togenerate one hash function.

A. Euclidean Distance Preserving as Variance Estimation

We commence by reviewing basics of p-stable distribution,and then describe how it can be used to preserve the originaldistance between data points. This process can be thought ofas estimating the variance of a specific distribution.

A random variable has a stable distribution if a linearcombination of independent copies of the variable followsa similar distribution. For a p-stable distributionD, givent real numbersb1...bt and random variablesX1...Xt whichare independently and identically drawn from distributionD,∑

i biXi will follow the same distribution as(∑

i |bi|p)1/pX ,

4

whereX is a random variable with distributionD andp is aparameter subject top ≥ 0 [16]. It has been proved that stabledistribution exists whenp ∈ (0, 2] [12]. Particularly, whenp = 1 andp = 2, the corresponding p-stable distributions areCauchy distribution and Gaussian distribution, respectively.

Let w denote ad-dimensional random vector whose en-tries are generated independently from a standard GaussiandistributionDs (with zero mean and unit standard deviation).Let vi andvj be two data vectors with dimensionalityd, andthe distribution ofwT vi − wT vj = wT (vi − vj) follows aGaussian distributionDg which has zero mean and variance‖vi − vj‖

2. Let W denote ad× r matrix whose each columnis a random vector which can be thought of a vector behavinglike w. Ther entries of the vectorWT (vi−vj) are independentof each other and followDg. This implies that for arbitraryWT (vi−vj), 1

r‖WT (vi−vj)‖

2 is an estimator of the varianceof Dg. We can get the expectation of the random variable1r‖W

T (vi − vj)‖2:

E[1

r‖WT (vi − vj)‖

2] = ‖vi − vj‖2 (1)

where‖·‖ is thel2 norm. Equation (1) also shows that this is anunbiased estimate. Furthermore, using the probability densityfunction of Gaussian distribution, we can get the variance ofthis estimator:

Var[1

r‖WT (vi − vj)‖

2] =2

r‖vi − vj‖

4 (2)

We observe that largerr leads to smaller variance and givesmore precise estimation. In LSH,r corresponds to the lengthof hash code. Therefore, equation (2) also explains why LSHperforms better with longer hash codes.

B. Learning Projection Vectors

The LSH scheme uses one random vector to generate onehash function (hash bit). Precise characterization of LSHrequires a large number of random vector samples, which leadsto long hash code. However, long hash code is less preferredin practice because it leads to low recall, sparse hash tableand decreased efficiency. An intrinsic solution to overcomethis disadvantage is to change the one-to-one correspondencebetween random vectors and hash bits. Different from LSH,we propose multiple locality sensitive hashing (MLSH) whichusesc different Gaussian random vectors to generate one bit.By using c × r random vectors, our MLSH generatesr hashbits. In contrast, by using the same number of random vectors,LSH results in longer code withc× r hash bits, which is lessefficient.

For a hashing scheme withr hash bits, our method can beimplemented through estimating the variance of the Gaussiandistribution Dg based onc × r random samples, which ismotivated by the principles described in Section III-A. LetQ be ad×c matrix whose each column is a Gaussian randomvector. If our hash function is constrained to be in a linearform, then for each hash function, our objective is finding ad-dimensional projection vectoru:

argminu

n∑

i,j

(‖QT vi −QT vj‖2 − (uT vi − uT vj)

2)2 (3)

By discarding the magnitude factor, we can assume thatu =Ql where l is a c-dimension unit vector,i.e., ‖l‖2 = 1. Sothe term‖QTvi −QT vj‖

2 − (lTQT vi − lTQT vj)2 is always

non-negative, and our objective becomes:

minl

n∑

i,j

(‖QT vi −QT vj‖2 − (lTQT vi − lTQT vj)

2) (4)

Proposition 1. Finding the optimal solution in problem (4) isequivalent to the maximization problem:

maxl

lTQTV V TQl

subject to ‖l‖2 = 1.(5)

whereV is a matrix with theith column beingvi.

Proof. The minimization problem in (4) can be transformedto the maximization problem as follows:

argmaxl

n∑

i,j

(lTQT vi − lTQT vj)2 (6)

The sum of the squared pairwise difference has a proportionalrelation with the variance. SupposeV is a matrix with theithcolumn beingvi, and we have:

n∑

i,j

(lTQT vi − lTQT vj)2 ∝ Var(lTQTV ) (7)

where Var(·) is the sample variance of elements in the vector.For the zero-mean data, Var(lTQTV ) = 1

n lTQTV V TQl.

Finally, we transform the initial objective (4) in terms ofthe optimization problem (5). The optimall is obtained bythe eigen-decomposition of the matrixQTV V TQ, where lis the eigenvector associated with the largest eigenvalue ofQTV V TQ. According to Proposition 1, this is also the optimalsolution of objective (4). Therefore, the approximate solutionof u for equation (3) is obtained byu = Ql. A d× r matrixU is then established, with its columns being vectors resultedfrom equation (5) by usingr different random matricesQseparately. We haveU = 1√

c×rU , and‖UT (vi − vj)‖

2 is anapproximation for the estimator with variance2c×r‖vi − vj‖

4

according to equation (2).

C. Minimizing the Error of Thresholding

For thed × r matrix U obtained in Section III-B, letUk

denote itskth column. The binary code for a feature vectorvican be obtained by applying sign function toUT

k vi. However,directly using sign(·) leads to considerable loss of accuracy inthe binary code. The quantization error of thresholding canbeestimated as:

n∑

i

r∑

k

(sign(UTk vi)− UT

k vi)2 (8)

The desiredU should have a small quantization error. Note thatin [6], Dataret al. quantized the real-valued output to discreteintegers to maintain accuracy. Nonetheless, binary codes aremore convenient for retrieval, which therefore, is adoptedinthis paper.

5

Algorithm 1: MLSH-ITQData: A d× n matrix V with each column being a

feature vector in the training set;The length of hashing codesr.

Result: A d× r projection matrixU .for m = 1 to r do

Generated× c matrix Q with each column being aGaussian random vector;Perform eigen-decomposition of the matrixQTV V TQ and letl equals the eigenvector associatedwith the largest eigenvalue;u← Ql;Um ← u;

endU ← 1√

c×r[U1, U2...Ur];

Solver × r orthogonal matrixR in (10) by the iterativeProcrustes method in [13];U ← UR.

Proposition 2. Given a projection matrixU and an arbitraryorthogonalr × r matrix R, U andUR have the same powerfor reconstructing the Euclidean distance.

Proof. For an arbitrary pair of feature vectorsvi and vj , wehave:

‖(UR)Tvi − (UR)T vj‖2 = ‖(UR)T (vi − vj)‖

2

= (vi − vj)T (UR)(UR)T (vi − vj)

= (vi − vj)TUUT (vi − vj)

= ‖UTvi − UT vj‖2 (9)

So the pairwise Euclidean distance of projection results underU andUR is the same.

According to Proposition 2,UR behaves the same asU . In the light of this observation, we aim to obtain anoptimal solutionR∗ to achieve the least quantization loss ofthresholding:

R∗ = argminR‖sign((UR)TV )− (UR)TV ‖2F (10)

where‖ · ‖F denotes the Frobenius norm.We follow the iterative method described in [13] to solve

objective function (10). In each iteration, it uses the classicOrthogonal Procrustes problem solution [40] to find an orthog-onal rotationR(i) (r×r orthogonal matrix) to align vector setUTV with sign(UTV ). After updatingU by UR(i), it startsa new iteration. Aftert iterations, the finalr × r orthogonalmatrix R∗ = R(1)R(2)R(3)...R(t) is obtained. The proposedmethod is summarized in Algorithm 1.

D. Constructing Multiple Hash Tables

For most data-dependent hashing methods, the limitationon the amount of hash functions leads to the incapabilityof constructing multiple hash tables. Since the matrixQ israndom, our method can construct multiple hash tables in

the same way as LSH. In this setting, the Hamming distancebetween binary codes ofvi andvj is:

dist(vi, vj) = mint=1..L

dHamming(Yt(vi), Yt(vj)) (11)

whereYt(vi) is the binary code ofvi in the t-th hash table.In [41], [42], methods are presented to build hash tables with

data-dependent strategy. However, these methods concentrateon the hash tables construction process. The idea is to trainthe hash functions of a data-dependent method with differentdata or parameters, which leads to the generation of differenthash tables. Our method, on the other hand, focuses on thehashing method itself and generates multiple hash tables byrandom vectors subject to p-stable distribution.

IV. SUPERVISEDHASHING BY INCORPORATING

SEMANTIC SIMILARITY

The method presented in Section III reconstructs the Eu-clidean distance in the original space and learns the hashfunction in an unsupervised manner. In many situations, theEuclidean distance between feature vectorsvi andvj does notreflect the real semantic similarity of objects. In this section,we present a supervised hashing method MLSH-SLP whichexplores supervised pairwise similarity. The whole procedureof this method and the relation with MLSH is shown inFigure 1.

A. Hashing Objective for Semantic Similarity

Let L(i) denote the class label of the objectvi andS denotea matrix whose (i,j)th entry Sij represents the supervisedsemantic similarity between two objectsi andj. Sij is definedas:

Sij =

{

1,L(i) = L(j);

0, otherwise.(12)

Our goal is to learn binary codes subject to the requirementthat neighboring samples in the same class are mapped tosimilar codes in the Hamming space. The neighborhood ofobject samples is measured in terms of semantic similarity.Inthis scenario, our method seeks anr-bit Hamming embedding1

Y ∈ {0, 1}r×n for n samples in the original space, and learnsr hash functionsh1,2,...,r : R

d → {0, 1}. Let yi denote theithcolumn ofY , i.e., the binary code for objectvi. We have anintuitive objective:

argminY

∑

i,j

Sij‖yi − yj‖2 (13)

subject to:yi ∈ {0, 1}r×1,n∑

i

yi =n

21r

where every column ofY should be independent of eachother. 1r is an r-dimensional vector of ones. The constraint∑n

i yi = n2 1r enables the data to be mapped into the hash

table uniformly. Minimizing (13) leads to smallSij beingassociated with large Hamming distance‖yi−yj‖

2, and vice-versa.

1Though slightly different, this definition is equivalent tothe previousbinary output{-1,1}.

6

Weiss et al. [7] have shown that a similar problem toEquation (13) is NP hard. Their solution is relaxation of theproblem to that of eigen-decomposition, which is based on thesimilarity measured in the original feature space. In contrast,we exploit semantic similarity and approximate a solutionusing the p-stable distribution theory and a coordinate descentmethod.

Let y(m) be an n-dimensional row vector denoting themth row of Y . We transform the original problem of learn-ing Y ∈ {0, 1}r×n to r subproblems of learningy(m) ∈{0, 1}1×n(m = 1, 2...r). Then each row vectory(m) couldbe learned separately form = 1, 2, ..., r through the samelearning strategy. Lety(m)

i denote theith element iny(m), werelax the original problem with a probabilistic form. Letp(m)

be ann-dimensional vector with theith componentp(m)i being

the probability fory(m)i = 1, i.e., the probability ofvi having

the binary output 1 with respect to themth hash function. Theexpectation ofy(m)

i is E[y(m)i ] = p

(m)i ·1+(1−p

(m)i )·0 = p

(m)i .

We formulate the objective function for themth subproblemof all the r subproblems as follows:

argminp(m)

∑

i,j

Sij‖p(m)i − p

(m)j ‖2 (14)

subject top(m)i ∈ [0, 1]

The method for obtaining optimalp(m) satisfying (14) isdescribed in the following two subsections.

B. Quasi Hash Bits

In this subsection, we present a strategy for initializing thehash probabilityp(m)

i , i.e., the probability for themth hashfunction to map the feature vectorvi to 1. We commenceby generating ad-dimensional vectoru by using the MLSHpresented in Section III. Then, for samplesi = 1, ..., n in thetraining set,p(m)

i is initialized as follows:

p(m)i =

1, uT vi > α+ ;

0, uT vi < α− ;

0.5, otherwise.

(15)

Here α+ and α−, which represent positive and negativethreshold parameters respectively, will be set empirically. Theinitialization strategy is developed in the light of the intuitionthat if the Euclidean distance between feature vectors of twoobjects is very large, then it is nearly impossible that theyare semantic neighbors. Note that we have already shown thatthe difference of projections onu could reflect the Euclideandistance between the original vectors. If‖pi − pj‖

2 > 0,which means sign(uT vi) 6= sign(uT vj), then|uT vi−uTvj | >α+−α−. Supposeα+−α− is large enough,Sij = 0 will bewith a high probability. When‖pi − pj‖

2 = 0, Sij will notinfluence the sum. Furthermore,uT vi has a zero mean whichapproximately satisfies

∑

i y(m)i = n/2, and the randomness

of u makesy(m)(m = 1, 2...r) independent of each other.Therefore, this partial solution satisfies the constraintsinequation (13).

For the time being, we set the hash bity(m)i for vi to be1

if p(m)i = 1, and set it to be0 if p

(m)i = 0. We refer to these

hash bits thus obtained as quasi hash bit. Furthermore, theremaining feature vectors associated with the hash probability0.5 tend to be less distinctive in terms of the projection onu,and we do not assign quasi hash bits to them.

C. Coordinate Descent

In this subsection, we use the coordinate descent methodto iteratively update the hash probabilities which are notassociated with quasi hash bits. In each iteration, we minimizethe objective function (14) by setting the derivative withrespect top(m)

i to be zero. Specifically, we treat onep(m)i

with the initial value 0.5 as a unique variable, hold all theother hash probabilities fixed, and updatep

(m)i as follows:

p(m)i =

n∑

j=1,j 6=i

Sijp(m)j

∑nk=1,k 6=i Sik

. (16)

Suppose we have in totaln′ hash probabilities which are notassociated with quasi hash bits. The coordinate descent methodbehaves in a way that one loop of iterations enumerate all then′ hash probabilities and then starts another loop of iterations.

Since Sij is always non-negative, and‖p(m)i − p

(m)j ‖2

is convex, problem (14) is a convex optimization problemwhich means it has global optimal solution. Furthermore, thesubproblem of the coordinate descent method is also convex,so the objective value

∑

i,j Sij‖p(m)i − p

(m)j ‖2 decreases after

each iteration. Figure 2 shows the convergence process ofthe optimization method for solving the problem (14) on theCIFAR-10 dataset. Details on this dataset are presented inSection V.

1 3 5 7 94.82

4.84

4.86

4.88

4.9

4.92

4.94

4.96

4.98

5x 10

4

# Outer loops

Obj

ectiv

e va

lue

Fig. 2. Numerical result of the convergency of the optimization process onthe CIFAR-10 dataset.

After p(m) converges, we get the refined hash probabilities.Then for one samplevi which is not assigned a quasi hashbit, we generate its binary code with respect to themth hashfunction as follows:

y(m)i =

{

1, p(m)i > 0.5;

0, otherwise.(17)

Repeat this procedurer times, r n-dimensional row vectorscan be generated. Finally, the{0, 1}r×n matrix Y can beestablished by concatenating ther n-dimensional row vectors.

7

D. Binary Codes for Queries

The scheme presented in Section IV-C only generates thebinary representations for samples in the training set. Inthis subsection, we investigate how to generate the codes ofa query. According to the definition of hashing, one hashfunction hm maps a samplevi in the original feature spaceto a binary valuey(m)

i ∈ {0, 1}. In this scenario, one hashfunction can be considered as a binary classifier. Therefore,generating the binary code for a query can be thought of asa binary classification problem. We use the training datasetconsisting ofvi for i = 1, · · · , n and the correspondingr-bitHamming embeddingY obtained in Section IV-C to trainrbinary classifiers. Themth binary classifier categorizes a queryinto the class with label 0 or 1, which is themth binary codefor the query accordingly. Therefore, it is reasonable to referto themth binary classifier as themth hash functionhm.

E. A Label Propagation View of the Proposed Framework

We consider one hash function as a binary classifier and thehash bits as the labels of samples. For normal classificationproblems, the labels of training samples are usually obtainedthrough human annotation. On the other hand, for a hashfunction, a sample is assigned a hash bit. Specifically, thecriterion for this assignment is based on equation (13), whichis intrinsically similar to that of label propagation. Differentfrom general label propagation that uses feature similarityto propagate the label, our method uses semantic similarityto propagate the hash bits. However, our method and labelpropagation share the common underlying principle that oneclassifier should assign the same class labels to neighboringsamples with a high probability. Therefore, we refer to ourmethod described in this section as multiple locality sensitivehashing with supervised label propagation (MLSH-SLP). Thedetail of the whole procedure is described in Algorithm 2.

V. EXPERIMENTS

A. Datasets and Experiments Setup

We evaluated the performance of the proposed methods onthree popular image datasets. These datasets vary in content,image sizes, class labels, and human annotations.

CIFAR-10 dataset [43] consists of 60,000 32×32 colorimages in 10 classes, with 6,000 images per class. The classesinclude airplane, automobile, bird, cat, deer, dog, frog, horse,ship, and truck. Figure 3 shows some sample images randomlyselected from each class.

MNIST database consists of 70,000 handwritten digitimages, including 60,000 examples in the training set, and10,000 examples in the test set. It is a subset extracted fromalarger set available from NIST. The images are28× 28 greyscale. This dataset has 10 classes corresponding to the 0∼9digits, with all images being labeled.

NUS-WIDE is a web image dataset created by the Labfor Media Search in National University of Singapore, whichcontains 269,648 images downloaded from Flickr. The ground-truth of these images are provided in multiple labels such thateach image is labeled as a vector of zeros and ones to represent

Algorithm 2: MLSH-SLPData: A d× n matrix V with each column being a

feature vector in the training set;A similarity matrix S;The length of hashing codesr.

Result: Binary codesY ;A set of r hash functionshm(·) for m = 1, 2, ..., r.

for m = 1 to r doGenerate a vectoru as described in Section III-B;

Initialize p(m)i according to (15);

for i = 1 to n doif p

(m)i = 1 thenAssign a quasi hash bit 1 tovi;

endif p

(m)i = 0 thenAssign a quasi hash bit 0 tovi;

endendwhile p(m) is not convergeddo

for i = 1 to n doif vi does not have a quasi hash bitthen

p(m)i ←

∑nj=1,j 6=i

Sijp(m)j∑

nk=1,k 6=i

Sik;

endend

endAssigny(m) according to (17);hm ←Classifier(V , y(m));

endY = [y(1)T , y(2)T , ..., y(r)T ]T .

whether it belongs to one of the 81 defined concepts. Eachimage can be assigned with multiple concepts.

We extracted different image features on each dataset due todifferent properties of corresponding images. For CIFAR-10,the images are too small to extract good scale invariant localfeatures such as SIFT [44]. Considering that images are in thesame size, we used a 512-dimensional GIST descriptor [3] torepresent each image. In MNIST, the digit in each image iswell aligned, so the gray values of each image can be treated asa 784-dimension feature vector. Since a major portion of pixelsare clean background pixels, each feature vector has a sparseform. Images in NUS-WIDE are in larger size and containlots of detail information. In the experiments, we used the500-dimensional Bag-of-Words [2] feature vector built fromSIFT descriptions for image retrieval.

The proposed MLSH-SLP method can work with variousclassifiers. In the experiments, we chose linear SVM as themodel of hash function in order to meet the efficiency require-ments of image retrieval. The linear model is efficient in theprediction phase, which is very important to the indexing time.In the implementation, we employed the LIBLINEAR [45]which has low time complexity and good classification accu-racy. The main parameters are set as default values providedby LIBLINEAR, i.e., cost C = 1, dual maximal violationtoleranceǫ = 0.1.

8

Fig. 3. Random samples from the CIFAR-10 dataset. Each row contains 10images of one class.

B. Evaluation Protocols and Baseline Methods

In the experiments, we evaluated the proposed MLSH-ITQand MLSH-SLP methods in both unsupervised and supervisedsettings. In the unsupervised setting, we used the Euclideanneighbors as the ground truth. Similar to [17], we used theaverage distance of all query samples to the 50th nearestneighbor as a threshold to determine whether a point in thedataset should be considered as a true positive for a query.In the supervised setting, we used class labels as the groundtruth. In the CIFAR-10 dataset and the MNIST dataset, eachimage has a single class label, then images in the same classare considered as the true neighbors to each other. While onNUS-WIDE dataset, we followed the protocol in [29] such thatthe ground truth is determined based on whether two samplesshare at least one semantic label.

We randomly chose 1000 samples from each dataset as thequery images, and used the rest of the dataset as the targetof the search. For MLSH-ITQ, we used all samples but thequery images in the dataset as the training set. We randomlyselected 2000 samples from each dataset for training MLSH-SLP because of its relatively high computational complexity.We used the same size of training sets as described in theiroriginal papers for all alternative methods.

We adopted the precision-recall curve to compare the over-all performance of all methods. In our experiments, it wascomputed by:

precision=Number of retrieved relevant pairs

Total number of retrieved pairs(18)

recall=Number of retrieved relevant pairs

Total number of relevant pairs(19)

For the given queries, we increase the Hamming radius from0 to r to generater + 1 pairs of precision and recall values,

then the precision-recall curve is plotted. As a complement,we also calculated the mean average precision (mAP), whichis the area under the precision-recall curve.

In practice, there are two major applications for the resultedhash binary codes, i.e. Hamming ranking and hash lookup.Hamming ranking compares the binary code of the query withall samples in the database, which leads to linear complexitybut can be efficient thanks to the efficacy of the comparisonoperator of binary codes. Hamming ranking is usually usedwith longer code length. Hash lookup constructs a lookup tablefor the database. With the binary code of a query, it retrievessamples that fall within a bucket of the Hamming radiusδ. Toguarantee the efficiency of retrieval, the lookup table shouldnot be too sparse and the binary code should be compact. Inour experiments, we also compute the mean precision underdifferent hash code lengths for the Hamming radiusδ and thetop k returned samples of Hamming ranking:

mean precision=

∑

iNumber of retrieved relevant samples for queryi

Total number of retrieved samples for queryi

Number of test samples(20)

If there is nothing in the buckets (i.e., no retrieved samples)for certain Hamming radiusδ and query sample, we considerit to be zero precision.

We compared our methods with some state-of-the-art unsu-pervised hashing methods, which include iterative quantizationbased on PCA (PCA-ITQ) [13], k-means hashing (KMH) [39],spherical hashing (SPH) [38], unsupervised sequential learninghashing (USPLH) [30], spectral hashing (SH) [7], and localitysensitive hashing (LSH) [6]. For supervised or semi-supervisedhashing methods, we evaluated iterative quantization based onCCA (CCA-ITQ) [13], semi-supervised hashing (SSH) [29]and semi-supervised sequential projection learning hashing(S3PLH) [30]. In these methods, LSH and our method MLSH-ITQ can directly construct multiple hash tables, we denotethem as LSH-m and MLSH-ITQ-m, respectively. A summaryon different properties of these methods is given in Table I.

Method Hash Function Learning Paradigm

PCA-ITQ [13] linear unsupervisedKMH [39] nonlinear unsupervisedSPH [38] nonlinear unsupervised

USPLH [30] linear unsupervisedSH [7] nonlinear unsupervised

LSH [6] linear (data-independent)CCA-ITQ [13] linear supervised

SSH [29] linear semi-supervisedS3PLH [30] linear semi-supervised

TABLE ISUMMARY OF PROPERTIES OF HASHING METHODS UNDER COMPARISON.

Through Table I, we observe that hash functions in PCA-ITQ, CCA-ITQ, LSH, SSH, USPLH and S3PLH have thelinear form of hash functions, which are the same as ourmethods. On the other hand, KMH, SPH and SH use nonlinearhash functions, but still achieve a constant time complexity forcomputing binary codes.

9

13

57

9

9

7

5

3

1

0

0.05

0.1

0.15

0.2

0.25

c

L

mA

P

(a) CIFAR-10

13

57

9

9

7

5

3

1

0

0.2

0.4

0.6

0.8

c

L

mA

P

(b) MNIST

Fig. 4. The mAP of different parameter settings for MLSH-ITQ-m on CIFAR-10 and MNIST.

C. Evaluation of Unsupervised Hashing Methods

Unsupervised hashing methods aim at finding the nearestneighbors of the query according to the Euclidean distance.They were originally developed for improving the time effi-ciency of nearest neighbor search. Because the class labelsare not available, the results depend on the distribution ofthe data. We chose two datasets,i.e., CIFAR-10 and MNISTwhich have distinctive data distributions. GIST descriptorswere extracted from CIFAR-10, which usually consists of non-zero real numbers. On the other hand, features extracted fromMNIST are sparse vectors, with most entries being zeros.After setting the ground truth by Euclidean distance as insection V-B, we compared LSH, SH, USPLH, SPH, KMH,PCA-ITQ and the proposed method MLSH-ITQ. There aretwo parameters in our method, the number of random vectorsfor one bit c, and the number of constructed hash tablesL. We set them with different values ranging from 1 to 9,and computed the mAP on each parameter setting. Figure 4shows the mAP of different parameter settings on CIFAR-10and MNIST. When using multiple hash tables, we returnedsamples within certain Hamming distanceδ in all L hashtables. Therefore, the recall always goes up with the increaseof L, but the precision does not. Because mAP is computedas the area under the precision-recall curve, too largeL willdecrease it. Largec can decrease the variance of the Euclideandistance estimator according to Section III-A, but may increasethe approximation error in equation (4). Because we use theeigen-decomposition based method solvel, too largec willdilute the information of the eigenvector corresponding tothelargest eigenvalue. We can see that too large or too smallvalues for bothL and c do not lead to good performance.Therefore, in the following experiments, we setL = 7 andc = 3 for both CIFAR-10 and MNIST.

Figures 5 and 6 show precision-recall curves for Euclideanneighbor retrieval on CIFAR-10 and MNIST, respectively. OnCIFAR-10, our method with multiple hash tables (MLSH-

ITQ-m) outperforms all alternative methods when the codelength is 32. When the code length equals 64 or 128, theperformances of MLSH-ITQ-m and PCA-ITQ are very close.Our method with single a hash table (MLSH-ITQ) outperformsall alternatives except PCA-ITQ, and its performance is veryclose to PCA-ITQ when the code length is greater than 64.For the alternative methods, LSH and SPH have significantimprovement when the code length increases. USPLH, SH,and KMH do not work well on CIFAR-10. On MNIST, KMHand USPLH have better performance and SPH performs theworst. Although both LSH and our method are based on the p-stable distribution, our method outperforms LSH significantlybecause of the data-dependent component. This superiorityismore obvious with short code length because our method takesthe data distribution into consideration.

To take the quantitative evaluation of the hash techniquesone step further, we used the mean precision and recall ofHamming radiusδ to evaluate different methods for hashlookup. Similar to many other hashing methods, we set theHamming radiusδ < 2, and computed the recall accordingto (19) and the mean precision according to (20). Figures 7and 8 illustrate these two measurements with respect to thelength of hashing codes, respectively. When the hash codelength r goes too large, the hash table becomes too sparse.For a given query, the buckets within Hamming radiusδ maycontain nothing, so the precision is looked as zero. Therefore,the performance with Hamming radiusδ may degrade whenthe code lengthr increases. It is clear that our single hash tablemethod MLSH-ITQ outperforms alternative single table meth-ods. We also observe that the MLSH-ITQ benefits from usingmultiple hash tables for hash lookup, because MLSH-ITQ-mhas demonstrated significant advantages over the alternativemethods.

Furthermore, we make an empirical comparison betweenMLSH-ITQ-m and LSH-m, under different number of hashtables. Figure 9 shows the mAP for LSH-m and MLSH-ITQ-m

10

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(a) 32 bits

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(b) 64 bits

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(c) 128 bits

Fig. 5. Precision-recall curves on CIFAR-10, using Euclidean ground truth.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ-m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(a) 32 bits

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(b) 64 bits

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(c) 128 bits

Fig. 6. Precision-recall curves on MNIST, using Euclidean ground truth.

16 32 48 64 800

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Length of hashing codes

Me

an

pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(a) CIFAR-10

16 32 48 64 800

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Length of hashing codes

Me

an

pre

cis

ion

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(b) MNIST

Fig. 7. Mean precision in Hamming radiusδ < 2 on CIFAR-10 and MNIST, using Euclidean ground truth.

11

16 32 48 64 800

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Length of hashing codes

Re

ca

ll

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(a) CIFAR-10

16 32 48 64 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Length of hashing codes

Re

ca

ll

MLSH-ITQ

MLSH-ITQ−m

KMH

SPH

PCA−ITQ

USPLH

SH

LSH

(b) MNIST

Fig. 8. Recall in Hamming radiusδ < 2 on CIFAR-10 and MNIST, using Euclidean ground truth.

1 3 5 7 9 110.12

0.14

0.16

0.18

0.2

0.22

0.24

0.26

Number of hash tables L

mA

P

MLSH-ITQ−m

LSH−m

(a) CIFAR-10

1 3 5 7 9 110.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

Number of hash tables L

mA

P

MLSH-ITQ−m

LSH−m

(b) MNIST

Fig. 9. The mAP for LSH-m and MLSH-ITQ-m with code length 48.

with the fixed code lengthr = 48. It is clear that our proposedmethod outperforms LSH-m. And in most cases, both methodshave better performance with more hash tables, and the mAPof MLSH-ITQ-m changes slightly whenL ≥ 5.

It is also clear from the experimental results that the data-independent methods perform better with longer code length.The reason for this is that the data-independent methodsrely on random projections. The larger number of projectionsthey have, the more precise they can recover the originaldistance. On the other hand, data-dependent methods capturethe distribution of data, so they usually have good performancewith relatively short code length.

D. Evaluation of Supervised Hashing Methods

In some practical applications, the neighborhood of a givenquery is not based on simple metric such as Euclidean dis-tance, but relies on the semantic similarity such as whethertwosamples belong to the same class. Therefore, we used the classlabels of image samples as the ground truth. We compared theproposed MLSH-SLP method with several alternative hashing

methods including LSH [6], SH [7], S3PLH [30], SSH [29],and CCA-ITQ [13].

Figure 10 shows the mean average precision under differ-ent code lengths on each dataset. The proposed MLSH-SLPmethod achieves the best results on all three datasets, andperforms better with longer code. S3PLH gets the second-best rank on both CIFAR-10 and MNIST. LSH performs thesecond-best on NUS-WIDE. For the other methods undercomparison, SH and SSH generate poor mAP, though SSHsometimes performs better than SH on NUS-WIDE. The mAPof CCA-ITQ degrades when the code length increases. Thereason may be that CCA-ITQ is based on the CanonicalCorrelation Analysis which usually has good performancewith low dimensional output. If the dimensionality of theoutput becomes higher, the useless dimensions of outputmay be introduced and will tarnish the useful dimensions. Itshould be noted that some methods do not generate consistentperformance on different dataset. On CIFAR-10 and MNISTdatasets, LSH has lower mean average precision than S3PLH,but on NUS-WIDE, LSH exceeds S3PLH. We can find that

12

32 64 128 2560.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

0.26

Length of hashing codes

mA

P

(a) CIFAR-10

32 64 128 2560.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Length of hashing codes

mA

P

(b) MNIST

32 64 128 2560.17

0.18

0.19

0.2

0.21

0.22

0.23

Length of hashing codes

mA

P

(c) NUS-WIDE

MLSH-SLP

CCA−ITQ

SSH

S3PLH

SH

LSH

Fig. 10. The mAP on CIFAR-10, MNIST, and NUS-WIDE, using class label ground truth.

32 48 64 80 96 112 128

0.2

0.25

0.3

0.35

0.4

Length of hashing codes

Mea

n pr

ecis

ion

(a) CIFAR-10

32 48 64 80 96 112 1280.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Length of hashing codes

Mea

n pr

ecis

ion

(b) MNIST

32 48 64 80 96 112 1280.17

0.18

0.19

0.2

0.21

0.22

0.23

0.24

0.25

0.26

Length of hashing codes

Mea

n pr

ecis

ion

(c) NUS-WIDE

MLSH-SLP

CCA−ITQ

SSH

S3PLH

SH

LSH

Fig. 11. Mean precision of top 500 Hamming neighbors on CIFAR-10, MNIST, and NUS-WIDE, using class label ground truth.

LSH performs better than many supervised methods on NUS-WIDE. This is because the Bag-Of-Words features used inNUS-WIDE can represent the content well,i.e., the Euclideandistance between the features can already give a good retrievalresult.

We also show in Figure 11 the mean precision of top 500Hamming neighbors. The code lengthr is in range[32, 128]. Itis clear that MLSH-SLP outperform the alternatives methodswith a large margin on all datasets. S3PLH performs wellon CIFAR-10, but has a low precision on NUS-WIDE. Themean precision of CCA-ITQ is lower with longer code. Ingeneral, CCA-ITQ and SSH perform better with compact hashcode than other methods which do not use the supervisedinformation.

Finally, we show samples of retrieved images on the CIFAR-10 dataset in Figure 12 with false positives labeled by redrectangles. This figure gives a qualitative evaluation of theretrieval performance of different methods.

E. Computational Cost

Table II shows the training and indexing time on CIFAR-10from each method. All experiments were implemented usingMATLAB, and ran on a PC with Core-i7 3.4GHZ CPU and16GB memory. LSH does not have a training phase because itis a data-independent method. We find that MLSH-SLP, KMHand USPLH take the highest training time. In the trainingprocedure of MLSH-SLP, doing propagation and training theSVM classifier cost the majority of time. Although the trainingphase of MLSH-SLP is time consuming, it can be boosted

with parallel computing because training each hash function isindependent. Almost all methods require only short indexingtime expect SH which has a more complex nonlinear hashfunction that takes a longer time to get the binary code.

When multiple hash tables are used, the training time andindexing time will beL times longer than the single hashtable version. This can also be reduced if we generate thehash functions and binary codes in parallel.

VI. CONCLUSION

In this paper, we have reviewed the properties of p-stabledistribution and shown how to incorporate it with trainingdata in data dependent setting. We have presented MLSH-ITQ which takes the distribution of data into consideration.It combines multiple random projections for minimizing thedifferences between pairwise distances of binary codes andoriginal vectors. Repeating the same procedurer times, wecan generate a vector inRr. We have also used an orthogonaltransformation to minimize the thresholding error, makingbinary codes accurately preserve the Euclidean distance. Com-pared with data-independent hashing such as LSH, this methodimproves the performance under compact binary codes. Inpractice, we can build multiple hash tables to improve theprecision and recall rate while most data-dependent hashingcan only use a single hash table. For ANN search based onsemantic similarity, we extend our method with supervisedinformation. We have proposed a supervised hashing method(MLSH-SLP), whose training procedure is similar to labelpropagation. For each bit, we use the p-stable properties to

13

query

MLSH-SLP CCA-ITQ SSH S3PLH SH LSH

Fig. 12. Qualitative results on CIFAR-10. We retrieved 25 Hamming neighbors of some query examples under 48-bit hashingcodes using each methods, andshow the false positives in red rectangle.

Methods 32 bits 64 bits 128 bits 256 bitsTraining Time Indexing Time Training Time Indexing Time Training Time Indexing Time Training Time Indexing Time

MLSH-SLP 58.19 0.19 115.87 0.24 230.78 0.39 463.13 0.63MLSH-ITQ 2.81 0.04 5.8 0.08 12.66 0.18 31.59 0.41

S3PLH 18.62 0.08 36.62 0.14 73.75 0.26 147.57 0.46USPLH 55.11 0.08 111.73 0.14 225.56 0.25 448.98 0.5

SSH 1.09 0.08 1.1 0.15 1.22 0.26 1.9 0.52LSH - 0.11 - 0.16 - 0.29 - 0.53SH 0.71 0.51 0.89 1.84 1.17 6.97 1.87 27.2

SPH 7.44 0.19 14.33 0.24 27.56 0.38 61.41 0.67KMH 117.02 1.22 128.44 1.26 156.88 1.35 215.37 1.57

PCA-ITQ 2.55 0.12 4.87 0.23 10.69 0.39 27.91 0.84CCA-ITQ 3.03 0.13 4.96 0.21 10.71 0.39 27.14 0.82

TABLE IITRAINING AND INDEXING TIME (SECONDS) ON CIFAR-10.

assign the quasi bits to a portion of samples in the trainingset, and then optimize the assignment of hash bits to theremaining samples according to the semantic similarity. Wehave evaluated these two hashing methods on three pub-lic image data sets. Compared with several state-of-the-arthashing approaches, the proposed methods have shown theirsuperiority. MLSH-ITQ with multiple hash tables has achievedthe best results for unsupervised cases and MLSH-SLP hasproduced the best performance for the supervised setting. Inthe future, we will expand this idea to other problems such asclustering or dimensionality reduction.

REFERENCES

[1] J. L. Bentley, “Multidimensional binary search trees used for associativesearching,”Communications of the ACM, vol. 18, no. 9, pp. 509–517,1975.

[2] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features:spatial pyramid matching for recognizing natural scene categories,” inProceedings of the IEEE Conference on Computer Vision and PatternRecognition, vol. 2, 2006, pp. 2169–2178.

[3] A. Oliva and A. Torralba, “Modeling the shape of the scene: Aholistic representation of the spatial envelope,”International Journalof Computer Vision, vol. 42, no. 3, pp. 145–175, 2001.

[4] R. Weber, H.-J. Schek, and S. Blott, “A quantitative analysis andperformance study for similarity-search methods in high-dimensionalspaces,” inProceedings of the International Conference on Very LargeData Bases, 1998, pp. 194–205.

[5] P. Indyk and R. Motwani, “Approximate nearest neighbors: towardsremoving the curse of dimensionality,” inProceedings of the AnnualACM symposium on Theory of computing, 1998, pp. 604–613.

[6] M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni, “Locality-sensitivehashing scheme based on p-stable distributions,” inProceedings of theAnnual Symposium on Computational Geometry, 2004, pp. 253–262.

[7] Y. Weiss, A. Torralba, and R. Fergus, “Spectral hashing,” in Proceedingsof the Neural Information Processing Systems Conference, 2008, pp.1753–1760.

[8] B. Kulis and T. Darrell, “Learning to hash with binary reconstructiveembeddings,”Proceedings of the Neural Information Processing SystemsConference, vol. 22, pp. 1042–1050, 2009.

[9] W. Liu, J. Wang, S. Kumar, and S.-F. Chang, “Hashing with graphs,”in Proceedings of the International Conference on Machine Learning,2011, pp. 1–8.

[10] M. Norouzi and D. J. Fleet, “Minimal loss hashing for compact binarycodes,” in Proceedings of the International Conference on MachineLearning, 2011, pp. 353–360.

[11] B. Kulis and K. Grauman, “Kernelized locality-sensitive hashing,”IEEE

14

Transactions on Pattern Analysis and Machine Intelligence, vol. 34,no. 6, pp. 1092–1104, 2012.

[12] V. Zolotarev,One-dimensional stable distributions. American Mathe-matical Society, 1986, vol. 65.

[13] Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantiza-tion: A procrustean approach to learning binary codes for large-scaleimage retrieval,”IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 35, no. 12, pp. 2916–2929, 2013.

[14] X. Zhu and Z. Ghahramani, “Learning from labeled and unlabeleddata with label propagation,” Technical Report CMU-CALD-02-107,Carnegie Mellon University, Tech. Rep., 2002.

[15] H. Yang, X. Bai, C. Liu, and J. Zhou, “Label propagation hashing basedon p-stable distribution and coordinate descent,” inProceedings of theInternational Conference on Image Processing, 2013.

[16] P. Indyk, “Stable distributions, pseudorandom generators, embeddingsand data stream computation,” inProceedings of the Annual Symposiumon Foundations of Computer Science, 2000, pp. 189–197.

[17] M. Raginsky and S. Lazebnik, “Locality-sensitive binary codes fromshift-invariant kernels,”Proceedings of the Neural Information Process-ing Systems Conference, vol. 22, pp. 1509–1517, 2009.

[18] M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniquesfor embedding and clustering,”Proceedings of the Neural InformationProcessing Systems Conference, vol. 14, pp. 585–591, 2001.

[19] Y. Bengio, O. Delalleau, N. L. Roux, J.-F. Paiement, P. Vincent, andM. Ouimet, “Learning eigenfunctions links spectral embedding andkernel pca,”Neural Computation, vol. 16, no. 10, pp. 2197–2219, 2004.

[20] D. Zhang, J. Wang, D. Cai, and J. Lu, “Self-taught hashing for fastsimilarity search,” in Proceedings of the International ACM SIGIRConference on Research and Development in Information Retrieval,2010, pp. 18–25.

[21] Y. Weiss, R. Fergus, and A. Torralba, “Multidimensional spectral hash-ing,” in Proceedings of the European Conference on Computer Vision,2012, pp. 340–353.

[22] P. Li, M. Wang, J. Cheng, C. Xu, and H. Lu, “Spectral hashing withsemantically consistent graph for image indexing,”IEEE Transactionson Multimedia, vol. 15, no. 1, pp. 141–152, 2013.

[23] F. Shen, C. Shen, Q. Shi, A. v. d. Hengel, and Z. Tang, “Inductive hash-ing on manifolds,”Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, 2013.

[24] I. T. Jolliffe, Principal Component Analysis. Springer-Verlag New York,1986, vol. 487.

[25] H. Jegou, M. Douze, C. Schmid, and P. Perez, “Aggregating localdescriptors into a compact image representation,” inProceedings of theIEEE Conference on Computer Vision and Pattern Recognition, 2010,pp. 3304–3311.

[26] W. Kong and W.-J. Li, “Isotropic hashing,” inProceedings of the NeuralInformation Processing Systems Conference, 2012, pp. 1655–1663.

[27] C. Strecha, A. M. Bronstein, M. M. Bronstein, and P. Fua,“Ldahash:Improved matching with smaller descriptors,”IEEE Transactions onPattern Analysis and Machine Intelligence, vol. 34, no. 1, pp. 66–78,2012.

[28] R. A. Fisher, “The use of multiple measurements in taxonomic prob-lems,” Annals of Eugenics, vol. 7, no. 2, pp. 179–188, 1936.

[29] J. Wang, S. Kumar, and S. Chang, “Semi-supervised hashing for largescale search,”IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 34, no. 12, pp. 2393–2406, 2012.

[30] J. Wang, S. Kumar, and S.-F. Chang, “Sequential projection learningfor hashing with compact codes,” inProceedings of the InternationalConference on Machine Learning, 2010, pp. 1127–1134.

[31] Y. Mu, J. Shen, and S. Yan, “Weakly-supervised hashing in kernelspace,” inProceedings of the IEEE Conference on Computer Visionand Pattern Recognition, 2010, pp. 3344–3351.

[32] W. Liu, J. Wang, R. Ji, Y. Jiang, and S. Chang, “Supervised hashing withkernels,” in Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, 2012, pp. 2074–2081.

[33] B. Kulis, P. Jain, and K. Grauman, “Fast similarity search for learnedmetrics,” IEEE Transactions on Pattern Analysis and Machine Intelli-gence, vol. 31, no. 12, pp. 2143–2157, 2009.

[34] R. Salakhutdinov and G. Hinton, “Semantic hashing,”InternationalJournal of Approximate Reasoning, vol. 50, no. 7, pp. 969–978, 2009.

[35] A. Joly and O. Buisson, “Random maximum margin hashing,” inProceedings of the IEEE Conference on Computer Vision and PatternRecognition, 2011, pp. 873–880.

[36] W. Kong, W.-J. Li, and M. Guo, “Manhattan hashing for large-scaleimage retrieval,” inProceedings of the International ACM SIGIR Con-ference on Research and Development in Information Retrieval, 2012,pp. 45–54.

[37] W. Kong and W.-J. Li, “Double-bit quantization for hashing,” in Pro-ceedings of the AAAI Conference on Artificial Intelligence, 2012.

[38] J.-P. Heo, Y. Lee, J. He, S.-F. Chang, and S.-E. Yoon, “Sphericalhashing,” inProceedings of the IEEE Conference on Computer Visionand Pattern Recognition, 2012, pp. 2957–2964.

[39] K. He, F. Wen, and J. Sun, “K-means hashing: an affinity-preservingquantization method for learning binary compact codes,” inProceedingsof the IEEE Conference on Computer Vision and Pattern Recognition,2013, pp. 2938–2945.

[40] P. H. Schonemann, “A generalized solution of the orthogonal procrustesproblem,” Psychometrika, vol. 31, no. 1, pp. 1–10, 1966.

[41] H. Xu, J. Wang, Z. Li, G. Zeng, S. Li, and N. Yu, “Complementaryhashing for approximate nearest neighbor search,” inProceedings of theInternational Conference on Computer Vision, 2011, pp. 1631–1638.

[42] X. Liu, J. He, and B. Lang, “Reciprocal hash tables for nearestneighbor search,” inProceedings of the AAAI Conference on ArtificialIntelligence, 2013.

[43] A. Krizhevsky and G. Hinton, “Learning multiple layersof featuresfrom tiny images,”Master’s thesis, Department of Computer Science,University of Toronto, 2009.

[44] D. Lowe, “Distinctive image features from scale-invariant keypoints,”International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110,2004.

[45] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin,“Liblinear: A library for large linear classification,”The Journal ofMachine Learning Research, vol. 9, pp. 1871–1874, 2008.