© 2013 SAP AG. All rights reserved. 1Public
© 2013 SAP AG. All rights reserved. 2Public
Agenda
Welcome
Agenda
• Introduction to Dobler Consulting
• Introduction to SAP Sybase IQ, Challenges and Trends, Key Capabilities
• What’s New In Version 16
• Q&A
Presenters
• Dan Lahl - Vice President of Product Marketing, SAP
• Joydeep Das - Vice President Product Management, SAP Database Management Portfolio
• Peter Dobler - CEO Dobler Consulting
• Allan Roeder - VP of Sales Dobler Consulting
Closing
© 2013 SAP AG. All rights reserved. 3Public
Introduction to Dobler Consulting
Dobler Consulting is a leading information technology and
database services company, offering a broad spectrum of
services to their clients; acting as your Trusted Adviser,
Provide License Sales, Architectural Review and Design
Consulting, Optimization Services, High Availability review
and enablement, Training and Cross Training, and lastly
ongoing support and preventative maintenance. Founded in
2000, the Tampa consulting firm specializes in SAP/Sybase,
Microsoft SQL Server, and Oracle.
Visit us online at www.doblerconsulting.com, or contact us
at 813 322 3240, or [email protected].
© 2013 SAP AG. All rights reserved. 4Public
Your Data is Your DNA, Dobler Consulting Focus Areas
Strategic
Database
Consulting
DBA
Managed
Services
Database
Training
Programs
SAP VAR D&T
Cross-Platform Expertise SAP Sybase® SQL Server® Oracle®
Dan Lahl
9/19/2013
SAP Sybase IQ 16
Tap Into Big Data at the Speed of Business
© 2013 SAP AG. All rights reserved. 6Public
Today’s topics
Introduction to SAP Sybase IQ
Challenges and Trends
Key Capabilities:
– Exploit the value of Big Data
– Transform businesses through deeper insights
– Extend the power of analytics across the entire enterprise
Customer and Ecosystem Success
© 2013 SAP AG. All rights reserved. 7Public
SAP Sybase IQ: smarter answers now
Up to 1,000x faster than
traditional DBMS
Handles volumes of ad
hoc queries in real time
Simple and complex
analytics on structured
and unstructured data
Speed & Flexibility
2x as many customers as
the next leading provider
96%+ customer
satisfaction rates
Consistently ranked as
EDW market leader by
leading analysts*
Mature and Proven
Compression reduces data
volume by up to 70%
Less storage and
maintenance required
Easy to lean and use –
requires minimal training
Lower TCO
Source: Magic Quadrant for Data Warehouse Data Management System, Gartner, January 31, 2013
Business Challenges and Trends
© 2013 SAP AG. All rights reserved. 9Public
What’s happening in the marketplace…
Exploding Data
Volumes
The Need for
Speed
Rising IT Costs
and Complexity
© 2013 SAP AG. All rights reserved. 10Public
Today’s challenges
Lost Revenues;
Lack of Insight
High Costs &
Complexities
Data Management
Challenges
Security
Slow Performance
© 2013 SAP AG. All rights reserved. 11Public
• Marketing analytics
Digital channelsTrack visits, discover best channel mix: e-mail, social media, and search
• Sales analytics
Deep correlationsPredict risks based on deal DNA (e-mails and meetings) and pattern match
• Operational analytics
Atomic machine dataAnalyze RFID, blogs, Short Message Service (SMS), and sensors — for continuous
operational inefficiency
• Financial analytics
Detailed simulationsLiquidity and portfolio simulations — for stress tests and error margins
Competitive advantage is now data-driven
© 2013 SAP AG. All rights reserved. 12Public
The data explosion continues
“Petabyte is the new Terabyte” - Forbes
Data volumes and applications in analytics environments are “growing up”
to Big Data
Source: Wikibon, Feb 2013, Big Data and Data Warehousing, www.wintercorp.com
$4.6B
Big Data
Apps
$6.9B
Key Capabilities
© 2013 SAP AG. All rights reserved. 14Public
SAP Sybase IQ 16
SAP Sybase IQ transforms the way
companies compete and win
through actionable intelligence
delivered at the speed of business
to more people and processes.
© 2013 SAP AG. All rights reserved. 15Public
Key CapabilitiesThe Value of SAP Sybase IQ 16
Extend the power of analytics across the entire
enterprise
Exploit the value of Big Data
Transform businesses through deeper insights
1
2
3
© 2013 SAP AG. All rights reserved. 16Public
Exploit the value of Big Data
Unconstrained, unlimited, fast
access
Flexible, expandable grid
architecture
Integral native frameworks
Cost-effective compression
© 2013 SAP AG. All rights reserved. 17Public
Transform business through deeper insights
Unified platform for all data
Low latency for immediate updates
Comprehensive answers using
massive volumes
© 2013 SAP AG. All rights reserved. 18Public
Extend the power of analytics across the entire enterprise
User driven self-service at “Google
speed”
Highly available enterprise data
Fast and efficient MPP and query
optimization
Secure data and systems
Customer & Partner Success
© 2013 SAP AG. All rights reserved. 20Public
SAP Sybase IQGlobal Presence in a Range of Data-Driven Industries
Since adding Sybase IQ, the demand
for reports has risen dramatically as
users realize just how much more
helpful the information is that we can
now provide. We receive requests for
10-20,000 more reports per month,
and ad hoc queries have grown from
5,000 to 10,000 per month.
Marc Guillard, DBA , BNP Paribas
Securities Services
“
”
“
”
“
”
SAP Sybase technology added a new
dimension to our business – we can
now report on any data our executives
need within minutes. Previously,
reports were impossible or took a full
day to generate.
Bonjan Sovilj, Research and Development
Manager, Mozzart
…in North America …in EMEA …in EMEA
I would venture to say SAP Sybase IQ
is the fastest in the industry. We load
almost 10 billion rows a month, a
terabyte a quarter. With SAP Sybase
IQ, it just flies.
Craig Silver, Senior Database Architect,
Nielsen Media Research
© 2013 SAP AG. All rights reserved. 21Public
Measurable benefits for our customers
75% 75% 800%Reduced daily DB
administration by
75%
Faster data loads -
increase productivity
during maintenance
window
Reduced BI storage
requirements by 75%
a ratio of 4 to 1
© 2013 SAP AG. All rights reserved. 22Public
2,500 + Customers
96% Customer
satisfaction rating
4,500 Installations
SAP Sybase IQ Market-Leading RDBMS Platform for EDW
Transforming companies through new insights.
SAP Sybase IQ 16: What’s New
Joydeep DasVice President, Product Management,
SAP DBMS Portfolio
© 2013 SAP AG. All rights reserved. 24Public
2009
Very large database (VLDB) platform foundation
2011
MapReduce API
2011
PlexQ MPP Foundation
2010
Full text search; Web 2.0 API
2009
In-database analytics
1H 2013
XLDB
Analytics
SAP SYBASE IQ 16
PATH TO ACTIONABLE INTELLIGENCE
© 2013 SAP AG. All rights reserved. 25Public
Ecosystem
SAP SYBASE IQ
A COMPREHENSIVE ANALYTICS PLATFORM
Most mature
column store
Comprehensive
lifecycle tiering
MPP queries, virtual
marts, and user scaling
High-speed
loads
Structured and
unstructured store
Comprehensive SQL
with OLAP
Built-in full-
text search
In-database analytics
with MapReduce
Web 2.0
APIs
Big Data open
source APIs
Optimized BI and
EIM
Development and
administration toolsPredictive analytics
Packaged ILM
applications
Bradmark,
Symantec,
Whitesands,
Quest, and
ZEND
SAS, SPSS,
KXEN, Fuzzy
Logix, Zementis,
and Visual
Numerics
BMMSoft,
SOLIX, and PBS
Sybase
PowerDesigner,
Sybase Replication
Server,
SAP and Panopticon
Application
services
DBMSHadoop
and R
SAP Sybase IQ 16
Architectural Details
© 2013 SAP AG. All rights reserved. 27Public
SAP Sybase IQ 16 Engine
Multip
lex g
rid a
rchite
ctu
re
Storage area network
Loadingengine
In-database analytics
Text search
Web-enabled analytics
Info
rma
tion life
cycle
ma
na
ge
me
nt
Low latency, write optimized store
Role-based access control
Communications and security
N-bit and tiered indexing
Column indexing
subsystem
Query engine
Column store
Aggressive scale out
Fullyparallel
Re
silie
nt
We
b b
ase
d a
dm
inis
tratio
n a
nd
mo
nito
ring
LDAP authentication
Hash partitioned tables
and data affinity
New Generation
PETABYTE SCALE store
New indexing technology for increased compression and fast, incremental batch loads
SAP SYBASE IQ 16
WHAT’S NEW!
High performance, fully parallel bulk loading
Enterprise-grade RBAC authorization and LDAP authentication
Comprehensive web-based monitoring and administration
Low latency, concurrent write optimized delta store
Re-engineered column store for extreme data volumes
Resilient Multiplex grid withstands network interrupts and server failures
Aggressively scaled out query engine
Intelligent query engine with data affinity for “shared nothing” performance on a flexible “shared everything” architecture
© 2013 SAP AG. All rights reserved. 28Public
SAP SYBASE IQ 16
INNOVATIONS FOR EXTREMELY LARGE DATABASES (XLDB)
Storage Architecture
• New generation column store
• New partitioning and compression
Query Processing
• Data affinity
• Aggressively parallel and distributed
Loading Engine
• Fully parallel bulk loading
• Real-time loading into delta store
System Reliability
• Grid resiliency
• LDAP and role-based security
SAP Sybase IQ
XLDB
Analytics
Petabytes Real-time
© 2013 SAP AG. All rights reserved. 29Public
SAP SYBASE IQ 16
NEW COLUMN STORE ARCHITECTURE
Value proposition
Enhanced compression
Storage savings
Improved I/O bandwidth
Architectural considerations
Support variable number of cells per page
Support various page formats within a column
High performance access paths
Even with variable length data, insert/update/delete
efficiently into an existing page
Richer metadata
Logical
page of
variable
sized cells
cell values
directory
Logical
page of
fixed sized
cells
Before
NOW…
© 2013 SAP AG. All rights reserved. 30Public
SAP SYBASE IQ 16
N-BIT DICTIONARY COMPRESSION
Value proposition
Reduced memory footprint
Improved effective I/O rates
More efficient table scans
Reduced disk space
Architectural considerations
N-bit Fast Project (FP) Indexes, instead of 1, 2 and 3-byte FP Indexes
Different data pages for same column can have different values of “N” for N-bit
No more requirement to rollover FP format for all column data
Column is N-bit by default, unless otherwise specified to be flat
Options provided to set threshold for rollover to flat (to prevent large dictionaries)
Options provided to prevent rollover to flat (to prevent long rollover time)
Compatibility mode allows the database to mimic IQ 15 rollover behavior
Raw Data = 400 MB; 1 Billion 4-byte integer values fn (N)
N Token Size Savings
2 (1B * 2 ) / 8 = 250MB 93.75%
3 (1B * 3 ) / 8 = 375MB 90.6%
4 (1B * 4 ) / 8 = 500MB 87.5%
…….
24 (1B * 24 ) / 8 = 3000MB 25%
2->3
3->4
4->5
5->6
6->8
8->10
10->12
12->16
16->21
21->24
© 2013 SAP AG. All rights reserved. 31Public
SAP SYBASE IQ 16
FULLY PARALLEL BULK LOAD
Value proposition
Improved performance
Maximize use of existing cores on the
machine
Dynamic load balancing
Architectural considerations
Load an index/column concurrently with
multiple threads
Remove all bottlenecks which contribute to
inefficient use of CPU and storage
Dynamically scale up and down degree of
parallelism depending on the workload
Before…
NOW
…
© 2013 SAP AG. All rights reserved. 32Public
SAP SYBASE IQ 16
SMALL BATCH LOAD PERFORMANCE
Value proposition
Improved performance of frequent, small batch
loads
Predictable performance of small batch loads:
performance is proportional to the size of the
data being loaded, not the table being
updated
Architectural considerations
Inserting into a large High Group (HG) b-tree
index is costly
HG index will have a tiered structure with a small
tree component and a large tree component
Small, batch loads into the HG index are written
to the small tree component quickly and
synchronously
The small tree component is periodically merged
into the large tree component as a background
task
Small tree
component
Btree-
based
index
Trickle
insert
Periodic
Merge
© 2013 SAP AG. All rights reserved. 33Public
SAP SYBASE IQ 16
HIGH VELOCITY DATA LOADING
Value proposition
Continuous analytics over operational data
High velocity, concurrent data modifications
Exploit large memory and core footprints
Architectural considerations
Write optimized in-memory In-memory RLV (Row-
level versioned) store
Row level locking, and statement snapshot
isolation
Low latency micro operations
In-memory RLV store has reduced compression,
no sorting, no indexing
Fully recoverable with dedicated transaction log
Asynchronous data transfer from In-memory RLV
store to IQ main store
Users choose which tables are In-memory RLV
tables
In-memory
RLV Store
RLV log on shared store
IQ main on shared store
Hybrid Table
Storage
Write ahead/commit
periodic merge
IQ query engine
IQ DML engine
RLV rows
main rows
Transaction Manager
Consistent View Manager
Version Manager
Lock Manager
Insert/Update/Delete
Co
nsisten
t View
global row
space
© 2013 SAP AG. All rights reserved. 34Public
SAP SYBASE IQ 16
QUERY SCALE OUT – HASH PARTITIONING
Value proposition
Gives best of both worlds of shared everything and
shared nothing
Decreases hardware needs and localizes processing
Architectural considerations
Data is automatically partitioned during loading with built-
in hash algorithms
Data is divided into persistent subsets
Reduces results sharing
More efficient CPU usage
Reduces instantaneous temp usage
Optimizer will use hash partitions for join and group by
when available
© 2013 SAP AG. All rights reserved. 35Public
SAP SYBASE IQ 16
QUERY SCALE OUT – DATA AFFINITY
Value proposition
Provides efficient utilization of cluster-wide
cache resources such as shared temp
Achieves ultra low-latency data access while
preserving elastic multiplex capabilities
Group by and Order by benefit
Architectural considerations
Affinity is automatically configured and used in
multiplex
Adapts to query workloads and self manages
Data must be hash or logically partitioned
Each partition is assigned to a specific node
Fact Dim
Cache Cache
© 2013 SAP AG. All rights reserved. 36Public
SAP SYBASE IQ 16
QUERY SCALE OUT – Query Runtime and DQP Optimization
Value proposition
Eliminates SMP (single node) and DQP
(distributed query processing) bottlenecks
Leverages large memory and scale out
Lowers shared temp and interconnect bandwidth
Architectural considerations
Takes place automatically as optimizer selects
best plan based on cost
For non-partitioned data, new Join and Group
algorithms reduce the amount of intermediate
results exchanged
Group ByQuery operator specialized for med cardinality
processing
Eliminate single-threaded hash
group-by bottlenecks
Improve DQP scale-out
Join New scale-out DQP merge join
PredicateOptimize in-
memory/cached processing
Eliminate intermediate
result bottlenecks
Improve DQP scale-out
SubqueryNew query
operators for improved
efficiency and scale-out
© 2013 SAP AG. All rights reserved. 37Public
SAP SYBASE IQ 16
LDAP AUTHENTICATION
Value proposition: Reduced TCO and improved
security
Enable customers to hook into existing enterprise infrastructures for
managing users and passwords
Enable central management of password complexity policies
Multiple domains and multiple LDAP servers
Architectural considerations:
Secure communication with LDAP server using TLS
Deployable across various vendor’s Directory Service that supports
Lightweight Directory Access Protocol (LDAP)
Support 24x7 operation: Automatic failover and failback
Efficient design for frequent, short-lived connections
No client side changes needed
SQL Anywhere, Sybase IQ, and ASE can share common user
repository
WebServer
© 2013 SAP AG. All rights reserved. 38Public
SAP SYBASE IQ 16
ROLE BASED ACCESS CONTROL
Value Proposition
Support separation of duties and principle of least privilege
Breakdown privileged operations into fine grained sets that can be individually
granted
Control over propagation of privileges
Who can grant which privileges
Complete backwards compatibility and clean migration
Stay competitive
Architectural Considerations
Support ANSI SQL role semantics, system defined roles and user defined roles
Minimum number of role administrators
Grantable system privileges for privileged database operations
Secure system stored procedures with SQL SECURITY INVOKER
Minimize performance impact by adding connection level cache mechanism
Same in-memory representation of RBAC structures: upgraded and non-
upgraded DBs
Restrict impersonation through SET USER to adhere to RBAC model
© 2013 SAP AG. All rights reserved. 39Public
SAP SYBASE IQ 16
SCALE OUT CLUSTER (MULTIPLEX) ENHANCEMENTS
• Shared System Temp- Reduces the size of local temporary store
- Simplifies sizing requirements for temp stores
- Opens up support for DQP on temp tables in future
• Logical Server – Login redirection- Zero changes on client side on dynamic changes to logical servers
- Single point to connection “redirector”
- HA with multiple servers in the connection string to prevent a single point of failure
• Cache Ejection Policy improvements for better cache hits
- Better infrastructure to track on disk changes and maximize cache hits to increase performance
• Global Transaction Resiliency
- Suspend/resume global DML transactions, with a timeout, during INC disconnect/reconnect, coordinator downtime
© 2013 SAP AG. All rights reserved. 40Public
SAP SYBASE IQ 16
SCALE OUT CLUSTER (MULTIPLEX) – GBL TRANSACTION RESILIENCY
Offline Coordinator
Writer1
Heartbeat
Client Reconnect
Reader2
Global R/W
transactions
suspend until
INC connections
reestablished
or timed out
Global R/W INC Connections
automatically reconnect and resume transaction
after a coordinator failover.
In most cases long-running loads will transparently resume.
Storag
e
Failover node restarts as Coordinator
© 2013 SAP AG. All rights reserved. 41Public
Summary
• Market-Leading product with tremendous momentum • 96%+ customer satisfaction rates• Pioneering Column-store with 10+ patents• World record holder in data loading• SAP Sybase IQ is used by twice as many companies as
the next leading pure column store DBMS provider• Focused sales and support teams• SAP commitment to product leadership
SAP Sybase IQ in the Context of RTDP
© 2013 SAP AG. All rights reserved. 43Public
Requirements for Real-Time Data-Driven ApplicationsDefining a Big Data-enabled Logical Data Warehouse
Business Applications
Endlessly expanding
repository for all kinds of data
Deep reference and
trend analysis
Instant classification of
streaming data
Real-time processing,
correlation, transactions, and
analysis
Data movement, cleansing, and
placement throughout the
architecture
Business Events
Metadata Management,
MDM, and Governance
© 2013 SAP AG. All rights reserved. 44Public
Enabling The Logical Data Warehouse With RTDPEngineered to Work Together – even with Big Data
HANA
IQ
Business Applications
ESP fast driver for HANA +
integration of ESP and
Hana Studios
Real-time Replication + ETL
to HANA and IQ
SQOOP to HANA and IQ
from/to Hadoop
Data Virtualization:
SAP Sybase IQ to
Hadoop to HANA
Query Integration: IQ
to Hadoop
SAP BW
Tightly integrated EDW
management with SAP
HANA and SAP Sybase IQ
Business Events
Da
ta S
erv
ice
s
ES
P
High Performance data
processing
with tight integration with SAP
BW, SAP Sybase IQ, ESP and
Data Services
© 2013 SAP AG. All rights reserved. 45Public
Isn’t it time to rethink your EDW strategy?Benefits of a complete Big Data Architecture
Accelerate AnalyzeAcquire
Effective, Rapidly
and Efficiently
acquire and
consolidate massive
amounts of diverse
and arbitrary data
Real-time response
time regardless of
data volume or data
location manage and
integrate massive
volumes of data
Achieve real results to
get the insights you
need with a variety of
means alone or in
combination
© 2013 SAP AG. All rights reserved. 46Public
Empowering you to Deliver Data Management
for the New Real-time World
Open APIs and Protocols
SAP Real-time Data Platform
Data Services Information ServicesCommon Services
TransactionReal-time
EDW
Integration Quality
MDM ILM
Mobile
Stream
Model
Manage
Metadata
Engineered to Work Together
Custom Mobile Analytics/BI Cloud Big Data
Thank you
© 2013 SAP AG. All rights reserved. 49Public
Offer from Dobler Consulting
For Qualified IQ installed base customers we will provide 4 hours of health
check / upgrade consulting
For Qualified New Customers we will provide 4 days of design and
optimization consulting
Please contact Allan Roeder at Dobler Consulting for more details.
Call (813) 322 3240 or email [email protected]
© 2013 SAP AG. All rights reserved. 50Public
Q & A
Please submit your questions through the questions panel
on the webinar control menu.
For more information on SAP Sybase IQ
Call 813 322 3240 or email: