Date post: | 29-Jan-2015 |
Category: |
Technology |
Upload: | search-technologies |
View: | 112 times |
Download: | 6 times |
The expert in the search space
Kamran Khan, President & CEO
Search: The Enabler for Big Data
Big Data for SearchHow can Big Data technologies help us to create better search systems?
Search for Big DataHow can search help to democratize
traditional Big Data?
“THE AVERAGE PERSON TODAY
PROCESSES MORE DATA IN A SINGLE
DAY THAN A PERSON IN THE 1500’S DID
IN AN ENTIRE LIFETIME.”
Search Technologies Overview
Herndon
San Diego
Ascot
Cincinnati
San Jose
Frankfurt
• The leading company dedicated to Enterprise Search & Big Data Solutions
• Implementation, Consulting, Managed Services, Technology
• 150 employees and growing
• Independent, working with all of the leading software vendors, and open source alternatives
500+ Customers
What is Big Data?
DATA
DATA
DATA
DATA
DATA
DATADATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
DATA
What is Big Data?
Too Big For A Single Machine Data Aggregation & Analysis Batch Processing
Message: Lots of Data “Big Data”
CLICK HERE TO VIEW VIDEO…
Tomorrow’s Enterprise Search Architecture
rrrrrConnector
StagingArea
Big Data
Search&
Analytics
ContentProcessing
MachineKnowledge
Sources
ContentSourcesSource
sSource
sSource
sContentSources
ContentProcessing
ADVANTAGES• Agility, flexibility, scalability• A platform for Big Data
enabled search and analytics applications• Fully embraces structured &
unstructured content Big Data for Search
• More diligent, and detailed Content Processing delivers better search, plus automated matching of CVs to job descriptions.
• Improved “fill-rates” add directly to the bottom line.
Example: Big Data for Search in Recruitment
Big Data (Hadoop)
New CVs & Jobs
Store vectors
Top 4K
vectors
Filter Query + Target SCIP
THESE ARE EXCITING TIMES
CLICK HERE TO READ MORE…
A Traditional Big Data Approach
Come up with the question
HDFS
Hadoop
A Traditional Big Data Approach Decide what data to use from the Data Warehouse
and other sources (Data Analyst) Write the required data to HDFS (Data Administrator) Run a batch job to produce results
Map Reduce
A Traditional Big Data ApproachRESULTS ARE VISUALIZED
PROS• Able to be very precise• Analysis is in highly-
trained hands
Traditional Method: PROS AND CONS
CONS• Need to know the
question ahead of time
• Limited bandwidth – requires specialist skills
• Lengthy iterations mean slow discovery, and lack of agility
HDFS
Hadoop
A Search ApproachWrite all meaningful data to HDFS
Search for Big Data
A Search ApproachRun Content Processing jobs to create schema-free content
objects (XML or JSON)
Search for Big Data
HDFS
Hadoop
Content Processing with M
ap Reduce
A Search ApproachFeed JSON Objects to a search engine
Search for Big Data
HDFS
Hadoop
Content Processing with M
ap Reduce
A Search ApproachUse search, supported by interactive user interfaces
Search for Big Data
PROS• Everyone knows search• Everyone can securely access
Big Data capabilities• Short, iterative, agile cycles —
Innovation• Unifies the analysis of structured
data & unstructured content• The democratization of
Big Data
Search For Big Data: PROS AND CONS
CONS• This is new, and evolving
• Not an out-of-the-box solution
• Expertise, experience, & not just technology, are needed
Insurance Fraud Example
• Search provides interactive exploration capabilities across a large set of records, and supporting unstructured content
• Investigators can interactively follow hunches, check things at low cost, find and pursue outliers that indicate fraudulent activity
Search for Big Data
CONCLUSIONWhichever way you look at it, technologies, expertise, and
experience from the enterprise search industry have a big part to play in the Big Data world.
As the leading services company in the sector, we look forward to further engaging with customers
on these projects.
Big Data for Search
Search for Big Data
The expert in the search space
Kamran Khan, President & [email protected]
QUESTIONS?
SEE US AT BOOTH 9