9/25/12   1  

An  Elephant  and  Six  Blind  Men  

An  Elephant  is  NOT  

•  Wall  •  Rope  •  Snake  •  Spear  •  Tree  •  Fan  

An  Elephant  is  …  an  Elephant  

Mobile  Social  Search  (MSS)  

is  NOT  Mobile    is  NOT  Social    is  NOT  Search  

Is  Mobile  Social  Search  

Changing  Times  in  Search  

Mobile  Social  Search:    Search  ‘inside’  problem  solving  in  real  ;me.  

Problem  Solving        

Search  

MSS:  A  Problem  of  Riches  

When  we  were  (data)  poor  –  we  searched.    

Now  that  we  are  (data)  rich  –  we  need  mobile  social  search.    

Search

Connecting

Then

Now

Then

Now

•  What  is  an  Italian  Restaurant?  – List  Italian  Restaurants  in  Brussels.  

•  Show  me  Italian  Restaurants?  – Use  Yelp  (or  any  other  social  approach)  and  distance  from  me  to  rank  them.  

•  Where  should  I  go  to  eat  Italian  food  NOW?  – Consider  Tme  taken,  current  ambiance,  and  quality  of  food  into  consideraTon.  

•  What  are  symptoms  of  Swine  Flu?  •  Is  there  a  major  Swine  Flu  outbreak  in  my  area?  •  You  are  likely  to  be  very  sick  with  extreme  case  of  Swine  Flu  soon,  your  doctor  is  ready  with  the  set  up.  

Social  Networks    

Connecting

People

Connecting People

And Resources

Social  Life  Networks  

 

Aggregation and

Composition

 

Situation Detection

Alerts

Queries

Information

9/25/12   14  

Concept  recogni;on  from  data  

15  

Environments  

Real  world  Objects  

SituaTons  

AcTviTes  

Heterogeneous  Media  

Single  Media  

SPACE  TIME  

Scenes  LocaTon  aware  

Visual  Objects  

Trajectories  

Visual  Events  

LocaTon  unaware  

StaTc   Dynamic  

Heterogeneous  Media  

Single  Media  360  K   11.4K  

3.4  K  LocaTon  aware  

LocaTon  unaware  

StaTc   Dynamic  

9/25/12   Proprietary  and  ConfidenTal,  Not  For  DistribuTon   16  

•  SituaTon:  An  acTonable  abstracTon  of  observed  spaTo-­‐temporal  characterisTcs.  

•  Allow  users  to  define  their  own  spaTo-­‐temporal  features  and  create  the  situaTon  detecTon  filters.  

 

9/25/12   17  

Swine  flu:  Situa;on  Segmenta;on  

into  ‘high’  and  ‘low  ’ac;vity  zones.  

9/25/12   18  Proprietary  and  ConfidenTal,  Not  For  DistribuTon  

(a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)  

d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)  

Representa;on  for  different  data  sources  into  a  common  spa;o-­‐temporal  format.    

Level  1:  Unified  representaTon  (STT  Data)  

Level  3:  Symbolic  rep.  (SituaTons)  

ProperTes

ProperTes

ProperTes

Level  0:  Raw  data  streams    e.g.  tweets,  cameras,  traffic,  weather,  …  

Level  2:  AggregaTon  (Emage)  

 

…  

STT  Stream  

Emage  

SituaTon  

9/25/12   21  

Billions  of  data  sources.    Selec;ng  and  combining  appropriate  sources  to  detect  situa;ons.    Interac;ons  with  different  types  of  Users  

 Decision  Makers                            Individuals    

Front  End  GUI

NewDataSource

NewQuery

E-­‐mageStream

E-­‐mage  Stream

E-­‐mage  Stream

Data  Cloud

Back  End  Controller

Stream  Query  Processor

Data  IngestorRegisteredData

Sources

RegisteredQueries

Raw  Spatial  Data  Stream

API  Calls

Raw  DataStorage

Personalized  Alert  Unit

AlertRequest

User  Info

9/25/12   22  

9/25/12   23  

S.  No   Operator   Input   Output  1   Selection  σ   Temporal    

E-­‐mage  Set  Temporal    E-­‐mage  Set  

2   Arithmetic    &  Logical⊕  

K*Temporal  E-­‐mage  Set  

Temporal  E-­‐mage  Set  

3   Aggregation  α   Temporal  E-­‐mage  set   Temporal  E-­‐mage  Set  4   Grouping  γ   Temporal  E-­‐mage  Set   Temporal  E-­‐mage  Set  5   Characterization  :  

• Spatial  φ  • Temporal  τ  

• Temporal  E-­‐mage  Set  • Temporal  Pixel  Set  

• Temporal  Pixel  Set  • Temporal  Pixel  Set  

6   Pattern  Matching  ψ  • Spatial  φ  • Temporal  τ  

• Temporal  E-­‐mage  Set  • Temporal  Pixel  Set  

• Temporal  Pixel  Set  • Temporal  Pixel  Set  

24  9/25/12   24  

Personal  SituaTon  

9/25/12   Proprietary  and  ConfidenTal,  Not  For  DistribuTon   25  

Situa;onal  controller    

• Goal    • Macro  SituaTon    • Rules  

Micro  event  e.g.  “Arrgggh,  I  

have  a  sore  throat”  (Loc=New  York,  Date=12/09/10)  

Macro  situa;on  

Control  Ac;on  “Please  visit  nearest  CDC  

center  at  4th  St  immediately”  

Date=12/09/10  

Alert  Level=High  

Level  1  personal  threat  +  Level  3  Macro  threat  -­‐>  Immediate  ac;on    9/25/12   26  

               

e.g.  High  Flu  risk        

+  

1)  Macro          situaTon  

Social  sensors  

Device  sensors  

Macro  sensors  

Personal  life  streams  

Profile/  Preferences  

+  

2)  Personalized  situaTon  

+  

Planetary  scale  sensing  

Personal  context  

Available  resources  

3)  Recommend                  AcTons  

Resource  data  

9/25/12   28  

Flood level - Shelter

Flood Level Shelter

Twitter

Classify (Flood level - Shelter)

Taking  personalized  acTons  

9/25/12   29  

•  Was  for  Archived  data  and  mostly  for  researchers  (using  Desktop).  

•  Currently:  Mobile  clients,  Local  data,  and  Social  Graph  (SoMoLo).  

•  Immediate  Future:  SituaTons  from  Real  Time  data  and  RecommendaTons.  

•  Future:  PredicTve  control  of  emerging  situaTons.  

Thanks  for  your  Tme  and  alenTon.  

For  quesTons:  jain@ics.uci.edu  

9/25/12   32  

Applica;on  scenarios  

¨  Business  decision  making:  Demand-­‐supply  analysis,  opening  a  new  store,  offer,…  

¨  Medical  :  Epidemic  monitoring,  Asthma,  polluTon  effect  miTgaTon  

¨  Disaster  relief:  (hurricane,  flood,  fire)  direcTng  people  to  appropriate  resources.  

¨  Traffic:  SuggesTng  best  routes