The Proven Analytical Platform for Big Data

February 2013

Kognitio is an in-memory analytical platform

Built from the ground-up to satisfy large and complex analytics on big data sets

A massively parallel, in-memory analytical engine that interoperates with your existing

infrastructure

Kognitio

•Privately held•Dev Labs in the UK •Leadership  in US•~100 employees

Core product:•MPP in‐memory analytical platform

•Built from the ground‐up to satisfy large and complex analytics on big data sets

Kognitio is focused on providing the premier high-performance analytical platform to power business

insight around the world.

Kognitio clients span the globe

The Kognitio Analytical Platform• Why an “analytical platform”?

– In the burgeoning “big data” ecosystem, the volume, velocity and variety of data require a new approach

• Disaggregation of persistent data storage and analytics• Variety of BI Tools (MicroStrategy, Tableau, MS Excel, etc.)• Introduce a new tier to accelerate, govern and increase flexibility

– Complement to Hadoop, EDWs, etc. • MPP in-memory structure enables fast ad-hoc reporting• Standard SQL, MDX, etc. to make Hadoop easy, consumable• Tight integration enables an “information anywhere” approach

Analytical Platform Reference Architecture

What is an “In-memory” Analytical Platform?

• A database where all of the data of interest or specific portions of the data have been permanently pre-loaded into a computers random access memory (RAM).

• Not a large cache– Data is held in structures that take advantage of the properties of

RAM – NOT copies of frequently used disk blocks– The databases query optimiser knows at all times exactly which

data is in memory and which is not

Kognitio Analytical Platform• A high performance in-memory analytical platform that

doesn’t require specialized servers

• Software– quick simple deployment on commodity hardware or Cloud

• Scalable– Linear scale-out through best of breed parallelism

• Powerful– Unrivalled MPP analytical performance– Harnesses all CPU cores made available

• Low TCO – Linux, commodity hardware, no special hardware needs– SQL relational core familiar to most DBAs

For Analytics, the CPU is King

• The key metric of any analytical platform should be GB/CPU– It needs to effectively utilize all available cores– Hyper threads are NOT the equivalent of cores

• Interactive/adhoc analytics: – THINK data to core ratios ≈ 10GB data per CPU core

• Every cycle is precious – CPU cores need to used efficiently– Techniques such as “dynamic machine code generation”

Makes in-memory databases go slowerMakes disk-based databases go faster

Careful – performance impact of compression:

Speed & Scale from “True MPP”• Memory & CPU on an individual server = NOWHERE near enough for big data

– Moore’s Law – The power of a processor doubles every two years– Data volumes – Double every year!!

• Every CPU core in• Every server needs to efficiently involved in • Every query

Every

– Data is split across all the CPU cores– All database operations are parallelised with no points of serialisation –

This is true MPP

• Combine the RAM of many individual servers• many CPU cores spread across• many CPUs, housed in • many individual computers (1 to 1000+)

Many

• The only way to keep up is to parallelise or scale-out

Free to use - Get started now

Try it now: http://www.kognitio.com/free

Kognitio Cloud

Cloud model enables multiple advantages

Kognitio Cloud is a ready-to-use analytical platform. Asecure Platform-as-a-Service (PaaS) available as either aPrivate or Public Cloud, it leverages the cloud computingmodel to make the Kognitio Analytical Platform availableon a subscription basis.

PRIVATE CLOUD• Could be referred to as an “exclusive” hybrid cloud offering

• Kognitio was the first to offer “Data-warehousing-as-a-Service” (DaaS) in 1993, managed services hosted solution model

• Designed for clients who require a secure, dedicated environment without the skills requirement and capital overhead associated with traditional, in-house analytical implementations

PUBLIC CLOUD• Ready-to-use in-memory analytical platform leveraging Amazon Web Services (AWS) Elastic Cloud Computing (EC2) infrastructure

• Based on hourly usage per CPU/server and TB of data

• Suitable for use cases with unpredictable usage patterns

• Automatically provisioning in minutes with pre-installed servers

• Elastic scalability (up and down) to meet compute demand

• Attractive to Line-of-Business functions• No software or hardware to buy, install, maintain or upgrade• Analysis projects can be brought to life quickly and easily

Fast execution / time-to-value

• PaaS model eliminates setup, maintenance and servicing• Enabling delivery of complex analytics to business users • “sandbox” environment for development and testing

Flexibility

• Avoid CapEx with only OpEx charges based on usage/subscription level

• Support and maintenance amortization across relevant contract periods

Lower costs

Analytics from the business user-down

2011 2010 Sep.3 Aug. Jul. Sep. Aug.3,443,873 8.1 382,009 401,951 391,878 351,696 369,199617,194 10.4 67,055 71,725 69,801 61,676 66,08565,237 1.0 7,671 7,892 7,422 7,357 7,61170,324 0.0 7,737 8,240 7,888 7,685 8,082226,261 5.8 24,764 26,196 25,973 23,288 23,722455,276 5.6 50,418 52,164 53,062 47,710 48,597446,918 3.5 48,368 51,797 51,160 46,166 49,84888,590 8.7 10,510 10,681 10,258 9,591 9,514279,985 13.2 31,390 31,889 28,478 28,266 28,282368,372 5.5 41,188 42,244 43,097 37,992 40,228

Not Adjusted9 Month Total 2011 2010*

Enables the Business: • Fast integration and time‐to‐value• Iterative “Sandbox” approach• Reduced risk

1. Understand the business problem2. Define the requirements

• Forecast ROIs and interation3. Perform a Kognitio Cloud Assessment4. Execute a cloud agreement with Kognitio5. Build the application6. Test and deploy the solution7. Ongoing development & improvement

IT

Business Analyst

Business User

Deploy with other technologies on AWS

• One click to launch!• Automatic deployment of Kognitio and BI

tools on Amazon Web Services• Self-Service BI NeutrinoBI at

nbi.kognitiocloud.com• Pre-loaded ready sample data in the

cloud for use and demonstration • Multi-node and single server self-paced

demonstrations • Videos, instructional information• Kognitio Community forum on LinkedIn

Public Cloud multi-node via CloudFormation• Kognitio configured as a multi-node deployment• Available as a trial platform on-demand• kognitio.kognitiocloud.com• Few steps to deployment

New! Kognitio version 8: Enabling and extending the Analytical Platform

External Tables

External FunctionsNot Only SQL

Hadoop Connector Other Connectors

Kognitio Storageas an External table

General Availability: June 2013

Create view image “name” as select “field1, field2” from “table” where date > 1/1/12

SelectMerchant_Group, to_char(Num_Accounts,'999,999') Num_Accounts,to_char(Num_Transactions, '999,999,999') Num_Trans,to_char(cast(Total_spend as dec(15,2)), '999,999,999') || ' K' otal_Spend_K

from (select MG.GroupDesc Merchant_Group, count(distinct Account_ID) as Num_Accounts,

count(*) as Num_Transactions, sum(Transaction_Amount) as Total_Spend from demo_fs.V_Fin_CC_Trans T, demo_fs.V_Fin_Merchant M, demo_fs.V_Fin_Merch_Group MG where T.Merchant_Category = M.CategoryNo and M.GroupNo=MG.GroupNo and upper(Location) in (select distinct upper(Town) from demo_fs.V_Fin_Postcodes where upper(Town) like '%LOW%')group by MG.GroupDesc ) SQ1order by Num_Accounts desc;

Kognitio Hadoop Integration• Developed in co-operation with Sears (Metascale)• More than just a connector – tight integration

– Hadoop does what it is good at – filtering data– Kognitio does what it is good at – complex analytics

Hadoop Cluster

Near-lineStorage

(optional)

Near-lineStorage

(optional)

Near-lineStorage

(optional)

Give me field1, field 2 from “file” where date > 1/1/12

Near-lineStorage

(optional)

Near-lineStorage

(optional)

Near-lineStorage

(optional)

Data

Kognitio Hadoop Connectors

HDFS Connector – fast load of complete files

• Connector defines access to HDFS file system• External table accesses row-based data

in HDFS• Dynamic access or “pin” data into memory• Complete HDFS file is loaded into memory• Data filtering requires data to be partitioned into

different files within Hadoop

Map Reduce Connector – filter from large files

• Connector uploads agent to Hadoop nodes• Query passes selections and relevant

predicates to agent• Data filtering and projection takes place locally

on each Hadoop node• Only data of interest is loaded into memory via

parallel load streams• Data can be filtered within a file

Not Only SQL

Kognitio External Scripts– Run third party binaries or scripts embedded within SQL

• Flexible framework to pass data to/from any executable or interpreter• Full MPP execution of Perl, Python, Java, R, SAS, etc.• Any number of rows in/out, partitioning controls

Not Only SQL: any language in-line

Kognitio External Scripts– Run third party binaries or scripts embedded within SQL

• Perl, Python, Java, R, SAS, etc.• One-to-many rows in, zero-to-many rows out, one to one

create interpreter perlinterpcommand '/usr/bin/perl' sends 'csv' receives 'csv' ;

select top 1000 words, count(*)from (external script using environment perlinterp

receives (txt varchar(32000))sends (words varchar(100))script S'endofperl(

while(<>){

chomp();s/[\,\.\!\_\\]//g;foreach $c (split(/ /)){ if($c =~ /^[a-zA-Z]+$/) { print "$c\n”} }

})endofperl'from (select comments from customer_enquiry))dt

group by 1 order by 2 desc;

This reads long comments text from customer enquiry table, in line perl converts long text into output stream of words (one word per row), query selects top 1000 words by frequency using standard SQL aggregation

Innovative client solutions

Orbitz leverages Kognitio Cloud to take large volumes of complex data, ingested in real time from web channels, demographic and psychographic data, customer segmentation and modeling scores and turn it into actionable intelligence, allowing them to think of new ways of offering the right products and services to its current and prospective client base.

PlaceIQ provides actionable hyper‐local Mobile BI location intelligence.  They leverage Kognitio to extracts intelligence from large amounts of place, social and mobile location‐based data to create hyper‐local, targetable audience profiles, giving advertisers the power to connect with consumers at the right place, at the right time, with the right message. 

Public Cloud

Private Cloud

Public Cloud

Software

Appliance

TiVo Research & Analytics 40 TBs of RAM that perform complex media analytics, cross‐correlating data from over 22 sources with set‐top box data to allow advertisers, networks and agencies  to analyze the ROI of creative campaigns while they are still in flight, enabling self‐service reporting for business users

The VivaKi Nerve Center provides social media and other analytics for  campaign monitoring and near real‐time advertising effectiveness.  This enables agencies in the Publicis Global Network to provide deep‐dive analytics into TBs of data in seconds

AIMIA provides self‐service customer loyalty analysis on over 24 billion transactions that are live in‐memory full volumes of POS data.  Retailers, Customer Packaged Goods companies and other service providers, provide merchandise managers with  “train‐of‐thought” analysis to better target customers.

Context for media analytics: • In‐memory analytical database for Big Data

• Correlate everything to everything

• MPP + Linear Scalability

• Predictable and ultra‐fast performance

• > 22 data sources

• Commodity servers/equipment

• Market‐available IT skills

• No solution re‐engineering

Solution Benefits– Reports allow advertisers, networks and agencies  to analyze the 

relative strengths and weaknesses of different creative executions, and how such variables as program environment, time slots, and pod position impact their ROI

– Enables self‐service reporting for business users

Mars, Inc.: “By using TRA to improve media plans, creative and 

flighting, Mars has achieved a portfolio increase in ROI versus a year ago of 25% in one category and 35% in a 

second category.”

Challenges– Expanding volumes of data– Few opportunities for 

summarization (demographics, purchaser targets, etc.)

– Data too large/complex for traditional database systems

– Need for simple administration

Analytics on tens of billions of events in tens of seconds with NO DBA

Loyalty marketing company that provides marketing and consulting services to retailers, service providers, and consumer packaged goods companies. Their Self-Service application offers “train-of-thought” analysis with near real-time data processing, enabling clients to better target customers.

Background

Case Study: AIMIAIn-memory analytics enable market basket analysis on with blazing speed

• Offer a near-time analytical environment where all EPOS transactions, not just sampled data, could be analyzed. (improve statistical confidence)

• Enable analysts to write a query and DB execute (no involvement from IT/DBAs)

Challenge

AIMIA lands a Kognitio Analytical Appliance they re-sell to each of their end-user clients, with years of full volume EPOS transactions + customer + product data (over 24 Billion transactions currently). All transactions are held in memory for complex basket analysis-type queries. S

olut

ion

Best-tuned Oracle RAC query ran in 25 min. same query Kognitio: 3 minutes!That was in the initial implementation, circa 2007. Today, average bundle of 12-18 queries runs in 90 seconds!R

esul

ts

Gartner: Kognitio is “visionary”

Strengths - Commentary • Consistent leadership with innovative pricing models• Pioneered data warehouse SaaS• Kognitio Cloud "on demand" cloud offering key for

growing clients • Unique ability to switch between Cloud and Platform • Meets Gartner Logical Data Warehouse concept • Innovative Hadoop integration• Great performance• Consistently satisfied clients with its great

performance• Makes it easier to use and run ad hoc queries• Recognized the shift from traditional warehousing• New features have extended capabilities to manage

external processes and data

What others say about Kognitio…

Think differently about business analyticsBusiness users require: • True ad-hoc analysis• Performance “at the glass”• Less reliance on IT

• Evolution required for Big Data Analytics: – Lower reliance on OLAP cubes and associated admin.– Stop building multiple dependent data marts, databases, etc.– Bring Hadoop in new use cases:

• “Dark Data”: Web, Social, History, etc.• Enable noSQL interoperability with existing tools

connect

www.kognitio.com

twitter.com/kognitiolinkedin.com/companies/kognitio

tinyurl.com/kognitio youtube.com/kognitio

NA: +1 855  KOGNITIOEMEA: +44 1344 300 770