+ All Categories
Home > Technology > Search: The Enabler for Big Data

Search: The Enabler for Big Data

Date post: 29-Jan-2015
Category:
Upload: search-technologies
View: 112 times
Download: 6 times
Share this document with a friend
Description:
Search Technologies' CEO Kamran Khan presents about Search for Big Data, and Big Data for Search at the May 2014 Enterprise Search and Discovery Conference in New York.
Popular Tags:
21
The expert in the search space Kamran Khan, President & CEO Search: The Enabler for Big Data
Transcript
Page 1: Search: The Enabler for Big Data

The expert in the search space

Kamran Khan, President & CEO

Search: The Enabler for Big Data

Page 2: Search: The Enabler for Big Data

Big Data for SearchHow can Big Data technologies help us to create better search systems?

Search for Big DataHow can search help to democratize

traditional Big Data?

“THE AVERAGE PERSON TODAY

PROCESSES MORE DATA IN A SINGLE

DAY THAN A PERSON IN THE 1500’S DID

IN AN ENTIRE LIFETIME.”

Page 3: Search: The Enabler for Big Data

Search Technologies Overview

Herndon

San Diego

Ascot

Cincinnati

San Jose

Frankfurt

• The leading company dedicated to Enterprise Search & Big Data Solutions

• Implementation, Consulting, Managed Services, Technology

• 150 employees and growing

• Independent, working with all of the leading software vendors, and open source alternatives

Page 4: Search: The Enabler for Big Data

500+ Customers

Page 5: Search: The Enabler for Big Data

What is Big Data?

DATA

DATA

DATA

DATA

DATA

DATADATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

Page 6: Search: The Enabler for Big Data

What is Big Data?

Too Big For A Single Machine Data Aggregation & Analysis Batch Processing

Message: Lots of Data “Big Data”

CLICK HERE TO VIEW VIDEO…

Page 7: Search: The Enabler for Big Data

Tomorrow’s Enterprise Search Architecture

rrrrrConnector

StagingArea

Big Data

Search&

Analytics

ContentProcessing

MachineKnowledge

Sources

ContentSourcesSource

sSource

sSource

sContentSources

ContentProcessing

ADVANTAGES• Agility, flexibility, scalability• A platform for Big Data

enabled search and analytics applications• Fully embraces structured &

unstructured content Big Data for Search

Page 8: Search: The Enabler for Big Data

• More diligent, and detailed Content Processing delivers better search, plus automated matching of CVs to job descriptions.

• Improved “fill-rates” add directly to the bottom line.

Example: Big Data for Search in Recruitment

Big Data (Hadoop)

New CVs & Jobs

Store vectors

Top 4K

vectors

Filter Query + Target SCIP

Page 10: Search: The Enabler for Big Data

A Traditional Big Data Approach

Come up with the question

Page 11: Search: The Enabler for Big Data

HDFS

Hadoop

A Traditional Big Data Approach Decide what data to use from the Data Warehouse

and other sources (Data Analyst) Write the required data to HDFS (Data Administrator) Run a batch job to produce results

Map Reduce

Page 12: Search: The Enabler for Big Data

A Traditional Big Data ApproachRESULTS ARE VISUALIZED

Page 13: Search: The Enabler for Big Data

PROS• Able to be very precise• Analysis is in highly-

trained hands

Traditional Method: PROS AND CONS

CONS• Need to know the

question ahead of time

• Limited bandwidth – requires specialist skills

• Lengthy iterations mean slow discovery, and lack of agility

Page 14: Search: The Enabler for Big Data

HDFS

Hadoop

A Search ApproachWrite all meaningful data to HDFS

Search for Big Data

Page 15: Search: The Enabler for Big Data

A Search ApproachRun Content Processing jobs to create schema-free content

objects (XML or JSON)

Search for Big Data

HDFS

Hadoop

Content Processing with M

ap Reduce

Page 16: Search: The Enabler for Big Data

A Search ApproachFeed JSON Objects to a search engine

Search for Big Data

HDFS

Hadoop

Content Processing with M

ap Reduce

Page 17: Search: The Enabler for Big Data

A Search ApproachUse search, supported by interactive user interfaces

Search for Big Data

Page 18: Search: The Enabler for Big Data

PROS• Everyone knows search• Everyone can securely access

Big Data capabilities• Short, iterative, agile cycles —

Innovation• Unifies the analysis of structured

data & unstructured content• The democratization of

Big Data

Search For Big Data: PROS AND CONS

CONS• This is new, and evolving

• Not an out-of-the-box solution

• Expertise, experience, & not just technology, are needed

Page 19: Search: The Enabler for Big Data

Insurance Fraud Example

• Search provides interactive exploration capabilities across a large set of records, and supporting unstructured content

• Investigators can interactively follow hunches, check things at low cost, find and pursue outliers that indicate fraudulent activity

Search for Big Data

Page 20: Search: The Enabler for Big Data

CONCLUSIONWhichever way you look at it, technologies, expertise, and

experience from the enterprise search industry have a big part to play in the Big Data world.

As the leading services company in the sector, we look forward to further engaging with customers

on these projects.

Big Data for Search

Search for Big Data

Page 21: Search: The Enabler for Big Data

The expert in the search space

Kamran Khan, President & [email protected]

QUESTIONS?

SEE US AT BOOTH 9


Recommended