Searching the Now

Josh Lucas

EarthLink

XTech 2007

Searching the Past

The Black Box

Under the Influence of Crawlers

Bigger is Better

The Rise of One Box

More, More, More

Faster, Faster, Faster

The Challenge

Who Am I ?

Where Am I?

What Time Is It ?

Let’s Build Something

How about an XTech Event search engine?

create_table :events do |t| t.column :location, :string t.column :date_start, :datetime t.column :date_end, :datetime t.column :summary, :string t.column :description, :string t.column :url, :string t.column :lat, :float t.column :lng, :float end

Real-Time Data

• Pingerati (http://www.pingerati.net)

• Microformats (http://www.microformats.org)

• Mofo (http://mofo.rubyforge.org)

<div class="en_session_title summary"><a href="http://2007.xtech.org/public/schedule/detail/161" class="url uid">Searching the Now</a></div><div class="en_session_roominfo"> <span class="en_session_topics"> <a href="http://2007.xtech.org/public/schedule/topic/1"> Applications</a> </span>

<span class="en_session_room location"> Louvre-Palais Royal</span>

<span class="uf_timing"> <abbr class="dtstart dtstamp"

title="20070518T0900"></abbr> <abbr class="dtend" title="20070518T0945">

</abbr></span>

</div>

require 'mofo' def index lat = 48.8583 lng = 2.2945 url = params[:url] cal = hCalendar.find url cal.each do |c| e = Event.new(:summary => c.summary, :url => c.url,

:lat => lat, :lng => lng, :date_end => c.dtend, :date_start => c.dtstart,

:location => c.location, :description => c.description).save end @num = cal.size end

Indexing and Searching

• Lucene (http://lucene.apache.org)• Solr (http://lucene.apache.org/solr/)• Ferret (http://ferret.davebalmain.com/trac)• acts_as_ferret (

http://projects.jkraemer.net/acts_as_ferret/wiki)

class Event < ActiveRecord::Base

acts_as_ferret :fields => [:summary, :description]

end

results = Event.find_by_contents(q)

Personalization

• Taste (http://taste.sourceforge.net/)

Location

• GeoKit (http://geokit.rubyforge.org/)

The Future

Thanks!

Josh Lucaslucasj@corp.earthlink.net