Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 217 times |
Download: | 0 times |
1
Extended SQL for Multimedia Queries
select <whats> from <whatr>where <preds>orderby <whato)groupby <whatg>
<whats> can be alphanum, images, functions<functions> can be numeric-valued, vector- valued (such as histograms), image-valued (such as extractors), aggregates,….<preds> can be hard constraints or similarity functions<whato) can be alphanum or similarities<whatg> can be alphanum or similarities
2
Queries from Current System
select A.id, A.img, simfMRI(A.img, B.img)from fMRI A, fMRI Bwhere B.id = 1 orderby simfMRI(A.img,B.img)
select A.id, I.img, S.probfrom fMRI A, simfMRI Swhere S.img = A.imgorderby S.prob
fMRI simfMRIid img img prob
*Assumes simfMRI is a global comparison of 2 fMRI images.
similarities of all images to aselected one withid = 1
3
Queries from Current System
select A.id, A.imgfrom fMRI A, fMRI Bwhere B.id = 1 and simfMRI(A.img, B.img)
select distinct A.id, A.imgfrom fMRI A, simfMRI Swhere S.fid = A.fid
fMRI simfMRIfid id img img prob
*Assumes simfMRI is a global comparison of 2 fMRI images.
similarities of all images to aselected one withid = 1
User query:
MystiQ query:
4
Potential Queries from Very Simple and Beyond
select simfMRI(A.img,B.img)from fMRI A, fMRI Bwhere A.id = 12 and B.id = 37
Return the similarity between the fMRI images ofpatient 12 and patient 37.
fMRI simfMRIid img prob single similarity value
5
Find pairs of patients in the database having the same age and similar fMRI images.
select A.id, B.id, P.age, simfMRI(A.img, B.img)from patient P, patient Q, fMRI A, fMRI Bwhere P.ID = A.id and Q.ID = B.id and A.age = B.ageorderby simfMRI(A.img, B.img)
patient fMRI simfMRIid age id img img img prob
pairs of database images inorder of similarity
Q3
6
Find pairs of patients in the database having the same age and similar fMRI images.
select A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI Bwhere P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI(A.img, B.img)
patient fMRI simfMRIid age fid1 fid2 prob
pairs of databaseimages
Q3
fid id img
User query:
select distinct A.id, B.id, P.age, from patient P, patient Q, fMRI A, fMRI B, simfMRIwhere P.ID = A.id and Q.ID = B.id and A.age = B.age and simfMRI.fid1 = A.fid and simfMRI.fid2=B.fid
MystiQ query:
Note: twopatients A.id,B.id may havemultiple similarimages. Theprobability is non-trivial.
7
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough.
select A.id, P.age, A.img, simfMRI(thisimg, A.img)where P.ID = A.id and simfMRI(thisimg,A.img) > threshorderby simfMRI(thisimg, A.img)
thisimg thresh patient fMRI simfMRIid age id img img prob
Q4
8
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough. select A.id, P.age, A.img
from patient P, fMRI Awhere P.ID = A.id and simfMRI(thisimg,A.img) > thresh and simfMRI(thisimg, A.img)
thisimg patient fMRI simfMRI
id age
Q4
User query:
MystiQ query:fid probfid id imgimg
select A.id, P.age, A.img, simfMRI(thisimg, A.img)from patient P, fMRI A, simFMRI Swhere P.ID = A.id and simfMRI(thisimg,A.img) > thresh and A.fid = S.fid
9
Given a query image called thisimg, return the id’sand similarities for database images that are similarenough.
select A.id, P.age, A.imgfrom patient P, fMRI Awhere P.ID = A.id and and HIGH(simfMRI(thisimg,A.img)) [weight=7] and simfMRI(thisimg, A.img) [weight=0]
thisimg patient fMRI simfMRI
id age
Q4
Better user query:
fid probfid id imgimg
10
Weights
I’m proposing extending the probabilistic predicatesand/or probabilistic tables with weightsE.g. FROM simfMRI S [weight=3.5]WHERE simfMRI(img1,img2) [weight=-0.5]
Default weight = 0.0Positive weight means the predicate is more importantNegative weight means it is less important.
11
High/Low Predicates
I’m also proposing introducing two newprobabilistic predicates:
High(value) = more “true” the higher “value” is
Low(value) = more “true” the lower “value” is
These predicates make sense only within a query:Different values are compared and the higher (lower)receive higher probability.
12
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc), simsignal(signal(A.img,loc), signal(thisimg,loc))from fMRI Awhere activation(A.img,loc) > threshorderby simsignal(signal(A.img,loc), signal(thisimg,loc)
thisimg loc thresh fMRI simsignal activation signal
signal prob(, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
fid id img
13
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc)from fMRI A, thisimg Twhere activation(A.img,loc) > thresh and simsignal(signal(A.img,loc), signal(T,loc))
thisimg loc thresh fMRI simsignal activation signal
signal prob (, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
User query:
fid id img
14
Given an image and a location, return id’s, images, and activation levels from database images that have high enough activation levels at that location in order bysimilarity at that particular location.
select A.id, A.img, activation(A.img,loc)from fMRI A, thisimg Twhere high(activation(A.img,loc)) [weight=10.0] and simsignal(signal(A.img,loc), signal(T,loc))
thisimg loc thresh fMRI simsignal activation signal
signal prob(, ) (,)
activation is a function that returns the mean activation level at a given location.signal is a function that returns the signal at a given location.locations denote particular voxels.
Q5
Better user query:
fid id img
15
Return ids and images for images (A) that have thehighest activation level of all database images (B) at a given location. For each of these, return id’s andimages (C) of all database images whose activationis above a threshold at that location, ordered bytheir signal similarity to A at that location.
select A.id, A.img, C.id, C.imgfrom fMRI A, fMRI B, fMRI Cwhere activation(A.img,loc) = max(activation(B.img,loc)) and activation(C.img,loc) > threshorderby simsignal(signal(A.img,loc),signal(C.img,loc))
loc thresh fMRI simsignal activation signalid img signal prob (, ) (,)
Q6
16
Return ids and images for images (A) that have thehighest activation level of all database images at a given location. For each of these, return id’s andimages (C) of all database images whose activationis above a threshold at that location, ordered bytheir signal similarity to A at that location.
select A.id, A.img, C.id, C.imgfrom fMRI A, fMRI Cwhere high(activation(A.img,loc)) [weight = 100.0]and high(activation(C.img,loc)) [weight = 0.0]and simsignal(signal(A.img,loc),signal(C.img,loc)) [weight=0.0]
loc thresh fMRI simsignal activation signalsignal prob (, ) (,)
Q6
User query:
fid id img
17
Extract and return a neighborhood of a given size abouta given location from a given query image.
select nbd(thisimg, loc, size)from thisimg
thisimg loc size
Simple example of an extraction function.
Q7
18
select A.id, A.img, I.loc, simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size)) as Sfrom fMRI A, intloc Iwhere A.img = I.img orderby S
Return id’s, images, and locations for images that havesimilar neighborhoods of a given size at that locationto neighborhoods of the query image thisimg at locationthisloc of the same given size.
thisimg thisloc fMRI intloc simnbd
img loc img nbd prob
The intloc relation contains the locations of interesting voxels in an image.This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.
thisimg
A
Q8
id img
19
select A.id, A.img, I.locfrom fMRI A, intloc Iwhere A.fid = I.fid and simnbd(nbd(A.img, I.loc, size), nbd(thisimg, thisloc, size))
Return id’s, images, and locations for images that havesimilar neighborhoods of a given size at that locationto neighborhoods of the query image thisimg at locationthisloc of the same given size.
thisimg thisloc fMRI intloc simnbd
fid loc prob img nbd prob
The intloc relation contains the locations of interesting voxels in an image.This is not straightforward, as we are computing similarity of neighborhoods of images, which are obtained through function calls, not through relations.
thisimg
A
Q8
User query:
fid id img
20
select A.id,A.img, I.locfrom fMRI A, SUR S, intloc I where A.img = I.img and simloc(I.loc, S.loc)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data.
fMRI SUR intloc simlocid img id loc img loc loc loc prob
Q9
21
select A.id,A.img, I.locfrom fMRI A, SUR S, intloc I where l.fid = A.fid and simloc(I.loc, S.loc)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data.
fMRI SUR intloc simlocid img id loc loc loc prob
Q9
User query:
fid loc prob
22
select A.id,A.img, I.loc, S.locfrom fMRI A, SUR S, intloc I where A.img = I.img and A.id = S.id and simloc(I.loc, S.loc) and simpeaks(signal(A.img,I.loc),S.signal)
Return id’s, images and pattern locations of database imagesthat have an interesting pattern at a location that is similar toone of the locations of language sites in the samepatient’s SUR data and whose fMRI signal at the fMRI locationhas a similar peak pattern to the SUR signal at its location.
fMRI SUR intloc simloc simpeaks
id img id loc signal img loc loc loc prob signal signal prob
Q10 We don’t understand this query yet