SAP HANA Data Warehousing Foundationa248.g.akamai.net/n/248/420835/cfaf8220e1d4cc817c... · SAP...

This is the current state of planning and may be changed by SAP at any time.

SAP HANA Data Warehousing Foundation Data Distribution Optimizer / Data Life Cycle Manager – DWF SP03

February, 2016


© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2Public

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a

purchase decision. This presentation is not subject to your license agreement or any other agreement

with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and SAP's

strategy and possible future developments are subject to change and may be changed by SAP at any

time for any reason without notice. This document is provided without a warranty of any kind, either

express or implied, including but not limited to, the implied warranties of merchantability, fitness for a

particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly negligent.


SAP HANA Data Warehousing Foundation Tools in detail

SAP HANA Data Warehousing Foundation 1.0

delivers specialized SAP HANA XS based

applications

Data Distribution Optimizer to plan, adjust and

analyze landscape reorganizations for SAP HANA

scale out systems

Data Lifecycle Manager to deliver a possibility to

archive / displace data from a SAP HANA persistency

to Dynamic Tiering, SAP IQ or Hadoop

Data Warehouse Monitor to provide a

comprehensive overview about current and past

activities in the data warehouse

Data Warehouse Scheduler to maintain

dependencies between single processes with the

focus to provision data warehouse models

SAP HANA

Data Warehousing

Foundation

Data Lifecycle Manager

Data Warehouse Monitor **

Data Warehouse Scheduler **

• DWF 1.0 SP00: DDO >= HANA Rev. 83

• DWF 1.0 SP01: DDO+DLM >= HANA Rev. 96

• DWF 1.0 SP02: DDO+DLM >= HANA Rev. 101

• DWF 1.0 SP03; DDO+DLM >= HANA Rev. 110 ** Outlook

Data Distribution Optimizer


Data Distribution Optimizer

(DDO)


Functionalities

SAP HANA Scale Out Landscape Overview with respect to system

configuration and data distribution

Specify different configurations for reorganization on different

levels of granularity (e.g. DB-Schema, Table, Table Group, Table

Subtype, Location Group)

Create, adjust and simulate different reorganization plans to

achieve an optimal data distribution in an interactive fashion.

Repeat the steps until a sufficient reorganization plan has been

created.

Graphical visualization of reorganization plans by comparing the

simulated with the actual system landscape

Export and import reorganization plans across system boundaries

(-> DDO remote mode deployment)

Execute & Schedule SAP HANA reorganization runs

Analyze reorganization plan steps and logs of planned and

executed SAP HANA landscape reorganization runs

Join Path Analysis to identify associated database tables in the

system and create a proposal for table grouping (SP03)

SAP Data Warehousing Foundation Data Distribution Optimizer


SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA)

• Identify associated database tables and make a proposal which are frequently used together are going to be placed

together (via Table Grouping)

• Use the proposal to assign the grouping information to the

database tables to significantly improve the placement of

tables, so that table joins are executed with an adequate level

of speed.

• Based on the data available in the statement cache

(HOST_SQL_PLAN_CACHE view of the _SYS_STATISTICS

schema) JPA uses currently the execution count of a

statement for prioritizing joins

• Only executed SQL statements that access tables or views will be

analyzed.

• SQL statements that haven't been executed yet or view definitions are

not considered for join path analysis. This means that depending on the

time of execution the results of the join path analysis may differ.

Table 1

Table 2 Table 7

Table 3

Table 8

Table Group 1 Table Group 2


SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA) - (Tabular Display)

Leverage the Grouping Granularity

Slider:

When increasing the granularity, the

groups get smaller (less tables within

a group) and the number of groups

increases.

When decreasing the granularity, the

groups get larger (more tables within a

group) and the number of groups

decreases.

The goal is to find a trade-off that has

enough groups so that the system can

be balanced well-enough, while at the

same time the number of cross-node-

joins is minimized.


SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA) - (Chart Visualization)

Adjustment to the Grouping

Granularity Slider would

apply for both Tabs:

The related Group(s) are

highlighted when selecting a

Group (here: Group 131)

representing the JPA

derived dependencies.

Further drill-down

capabilities are supported

within the Bubble-Chart.

The Tables + Table Size

listed in the “Tables in

Group xxx” box are derived

based on the selected

Group in the Bubble-Chart


Planned Innovations Future DirectionToday

SAP Data Warehousing Foundation Product Roadmap Overview – Data Distribution Optimizer

Future innovations

• Further enhancements of DDO

• Support and optimize

table partitioning in native

SAP HANA cases

• Flexible management of hosts

• Deeper insights and simulations

for data distributions• Additional KPIs

Q4 / 2015

SAP DWH Foundation 1.0 SP03

Data Distribution Optimizer (DDO)

• Deeper insights and simulations

for data distributions

• Expected duration

• Modification of proposals

• Improved Plan Generation runtime

• Workload analysis (e.g. plan cache

analysis)

• Analysis of SQL plan cache / participating

tables

• Improvements to the simulation of

landscape reorganization runs

Q2 / 2016

Upcoming planned release

Data Distribution Optimizer (DDO)

• Parametrization of JPA (e.g.

SCHEMA)

• Integration of JPA with Plan

Generation

• Optimal Group Granularity Detection

• Maintenance of Table Group

assignments for native HANA

Datamarts


Data Lifecycle Manager

(DLM)


SAP Data Warehousing Foundation DLM SPS03 - Graphical Exploration Module (Drill Down, Top 10)

• If the utilization forecast on the Home screen indicates that the utilization of one or more nodes could become

critical over time, you can use the exploration function to establish which tables are appropriate candidates for

data relocation.

Context

With a two-step approach, you can first

perform graph-based exploration on

different levels. With a sunburst chart, you

can drill down to information on different

levels, host, schema, or table level for

example. You can thus identify the ten

largest tables of all the tables in the

system, or the ten largest tables in a

given schema on a given host for

example. In a second step, you can use

form-based exploration to see how data

of a selected SAP HANA table is spread

according to a selected column filter. With

this insight, you can derive meaningful

business rules for relocating the data.


SAP Data Warehousing Foundation DLM SPS03 - Home Screen: KPIs

• Additional KPI: Forecast „HANA: Disk“

• Forecast „HANA: Disk“ in 100 days vs.

current utilization in addition to

Forecast „HANA: Memory“ in 100

days vs. current utilization


SAP Data Warehousing Foundation DLM SPS03 - Manage Storage Destinations

A storage destination specifies an instance of the storage stack which data can be relocated to with Data Lifecycle Manager.

Supported storage destination types are SAP HANA Dynamic Tiering, Spark SQL (DESTINATION) or SAP IQ for example. For

deletion of data, Data Lifecycle Manager offers storage destination type Deletion Bin.

A storage destination is assigned to a lifecycle profile. It is used there to define to which destination persistence the data will be

relocated to.

The Manage Storage Destinations screen of the Data Lifecycle Manager is split up into two main areas:

On the left side of the Storage Destinations area, you see a list of all available storage destinations. Here, you can search for

storage destinations by storage destination name or the user who last changed the destination. You can add new destinations or

delete destinations that are not needed anymore.

Note:

The instances of the storage stack have

to be installed and configured properly

before you can use them for defining

storage destinations.


SAP Data Warehousing Foundation DLM SPS03 - Supported Storage Destination Types

Storage Destination Type Restrictions

SAP HANA Dynamic Tiering Local The following factors prevent lifecycle profile from being activated:

• Source tables with keys of type

VARCHAR/CHAR/VARBINARY/BINARY and length greater than 255

• Source tables with keys of type NVARCHAR/NCHAR and length

greater than 85

• Source tables of type row

• Source tables with columns with data type BLOB

• Source tables without a primary key

SAP IQ over SDA

Spark SQL (DESTINATION) The following factors prevent the lifecycle profile from being activated:

• Source tables with columns with data type TEXT, SHORTTEXT,

BINTEXT, BLOB, CLOB, TIME, VARBINARY, NCLOB, or ALPHANUM

Deletion Bin Destination The following factors prevent the lifecycle profile from being activated:

• Source tables without a primary key


SAP Data Warehousing Foundation DLM SPS03 - Hadoop Storage Destination

Depending on the type of stores you would like to use for relocating data:

• The SAP HANA Spark Controller has been installed and configured, and added as Remote Source

to SAP HANA. The SAP HANA system has been configured for Spark connectivity.


SAP Data Warehousing Foundation DLM SPS03 - Hadoop Storage Destination (cont.)


SAP Data Warehousing Foundation DLM SPS03 - Manage Modeled Persistence Objects

• Modeled persistence objects help you treat multiple source persistence objects, like tables, jointly in matters of

data relocation. You can define a source persistence object from multiple SAP HANA tables. Such a modeled

persistence object can be used in a lifecycle profile. The relocation rule in this case can be defined for common

columns, which have been selected during definition of the modeled persistence object.

Note:

Modeled Persistence Objects are supported for storage destination types

SAP IQ over SDA and SAP HANA Dynamic Tiering.


SAP Data Warehousing Foundation DLM SPS03 - Destination Persistence Modeled Persistence Objects

Manage Lifecycle Profile –

Destination Persistency:

Visibility of the Data Flow(s) of the

defined Modeled Persistence

Objects SAP HANA Table Group

(here: YAM_ORDER_HDR +

YAM_ORDER_ITM)


SAP Data Warehousing Foundation DLM SPS03 - Rule Editor: Manage Modeled Persistence Objects

When you create and execute a lifecycle profile

for a modeled persistence object, the following

applies for all tables of the modeled object:

• A common relocation rule will be defined and

executed.

• A common clash strategy and packet size applies.

• A single relocation procedure will be generated.

• For all tables corresponding target tables will be

generated.

An example for a modeled persistence object is a

table group. A table group is a set of SAP HANA

tables with common columns. Table groups are

supported for storage destination types SAP IQ

over SDA and SAP HANA Dynamic Tiering Local.


SAP Data Warehousing Foundation DLM SPS03 - Import / Export of DLM Entities

• Data Lifecycle Manager entities like storage destinations, modeled persistence objects, or lifecycle

profiles can be described in JSON format. For transport or backup purposes, you can export and

import the entities in JSON format.


SAP Data Warehousing Foundation DLM SPS03 - Lifecycle profile: Nominal key

• For a source persistence type SAP HANA table, you specify a schema and table name for example.

Additionally, for tables without a primary key, you can specify a nominal key, which is treated like a

primary key by Data Lifecycle Manager. Using the nominal key, you can activate a life cycle profile

for a table without primary key.


Planned Innovations Future DirectionToday

SAP Data Warehousing Foundation Product Roadmap Overview – Data Lifecycle Manager

Future innovations

• Integration with SAP HANA Vora

• Pre-delivered Application DLM

templates

• Support for multi-tiering

• Integrated lifecycle management

• Exploration module based on SQL

Query Access Cache

• Parallelization of relocation packet

processing

• Support for HANA CDS Entities as

source persistence object

• Automated substitution of data

foundation in Calculation Views

(table replaced by generated view)

Q4 / 2015

SAP DWH Foundation 1.0 SP03

• Support for Table Groups

• Generate DLM profiles for a predefined

set of tables (table group)

• Graphical Exploration module for HANA

Table

• SPARK (SQL): HADOOP as storage

destination

Q2 / 2016

Upcoming planned release

• Table Hierarchy support (Foreign

Key Relationship)

• Rule Editor enhancements

(Semantic checks / Customer exit)

• Enhanced support for Table

Hierarchies and Table Groups for

certain storage destinations

• Feature completeness for HADOOP

as storage destination (packet size,

clash strategy)

• Adapt Relocation Agent module for

Table Groups & Table Hierarchies

for supported persistent storage

destinations


Appendix


1. Data Warehousing Foundation (DWF) SAP Help Page : http://help.sap.com/hana_options_dwf

2. SAP HANA DWF in SCN: http://scn.sap.com/docs/DOC-62482

3. Product Availability Matrix (PAM): https://support.sap.com/release-upgrade-maintenance/pam.htm

4. SWDC: https://support.sap.com/software/installations

1. -> A – Z Index -> SAP HANA Data Warehousing Foundation

5. Support / Incident Components

1. Component HAN-APP-DWS - for all SAP HANA Data Warehousing Foundation Option issues.

2. Component HAN-APP-DWS-DDO – for DDO related issues

3. Component HAN-APP-DWS-DLM – for DLM related issues

6. SAP HANA Academy Channel - Data Warehousing Foundation (DWF)

https://www.youtube.com/playlist?list=PLkzo92owKnVxZcLRGmfZS-bdlIFYRQ39R

7. DDO System Meta-Data Extraction: SAP Note 2138117

8. SAP DWF 1.0 Release SAP Note 2092669

9. SAP DWF 1.0 SP03 DDO Patch 1: 2259749

10.SAP DWF 1.0 SP03 DLM Patch 1: 2259304

SAP Data Warehousing Foundation Additional Information on DWF

http://help.sap.com/hana_options_dwf

http://scn.sap.com/docs/DOC-62482

https://support.sap.com/release-upgrade-maintenance/pam.htm

https://support.sap.com/software/installations

https://www.youtube.com/playlist?list=PLkzo92owKnVxZcLRGmfZS-bdlIFYRQ39R

http://service.sap.com/sap/support/notes/2138117




© 2015 SAP SE or an SAP affiliate company. All rights reserved.

Thank you


© 2015 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate

company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and

services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as

constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop

or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future

developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time

for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-

looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place

undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

http://global12.sap.com/corporate-en/legal/copyright/index.epx

Date post:	29-Mar-2018
Category:	Documents
Upload:	dangbao
View:	235 times
Download:	15 times

SAP HANA Data Warehousing Foundationa248.g.akamai.net/n/248/420835/cfaf8220e1d4cc817c... · SAP...

Documents