Guidelines ETL HANA v3 (1)

7/21/2019 Guidelines ETL HANA v3 (1)

http://slidepdf.com/reader/full/guidelines-etl-hana-v3-1 1/59

Produced by Wellesley Information Services, LLC, publisher of S

Guidelines for Extracting and Loading S

Non-SAP Data into SAP HANA

$r. %&arne %er#Comerit



In 'his Session

• We (ill e)plore SAP $ata Services and ho( to load information

*A+A• Learn ho( to create transformations, mer#es, and &oins

• )plore ho( to use %W e)tractors to load data into SAP *A+A (

Services

• We (ill discuss the best practices for modelin# your $W by vir

of your $W• At the end of this session, you (ill no( ho( to load SAP and n

into *A+A usin# SAP $ata Services and ho( to model you data

layer



Introduction 0 $r. %er#



What We1ll Cover

• Loadin# data to *A+A sin# SAP $ata Services

•

Levera#in# %W )tractors in *A+A, %W and in $ata Services• Wrap/up



SAP $ata Services 3 4vervie(

• SAP $ata Services is an 'L 'ool for enterprise information mana#ement p

solutions for5

$ata inte#ration and transformations

$ata 6uality

$ata profilin#

'e)t data processin#

SAP $ata Services transforms, refines, and delivers

trusted data for data tar#et database



What Are $ata Services %atch 7obs8

• %atch &obs are used for 'L5

)tract data from one or many sources

'ransform data to meet the or#ani-ation1s

business re6uirements

Load the processed data to a location for use



Step/by/Step5 Creatin# %atch 7obs

!. Create a ne( pro&ect and #ive it a

relevant pro&ect name

2. 9i#ht/clic on the pr

a ne( batch

'he practice of #ivin# relevant names to your pro&ects andbatch &obs is useful for or#ani-ation purposes



Step/%y/Step )ample5 Loadin# :lat :iles to SAP *A+A



4ther Available $ata Sources

se the local ob&ect library to find e)istin# data sources under the

;$ataStore< cate#ory

'here are many other data sources that can be used in $ata Services

=ou can upload more files under the ;:ormat< cate



:ormattin# the :lat :ile to be loaded

$ate fo

match d

>. In the ;:ile :ormat ditor< popup, fill in the appropriate fields

Semi column field separation (as defau

my data is tab/delimited so I must chan



$efinin# 'able :ields

?. nter the field properties

+otice the updated v



Previe( $ata before =ou Load it

". In the 9epository, under :ormat, ri#ht/clic and select

;@ie( $ata< to previe( the ne(ly added data source

'his allo(s you to chec if the source da

(ithout error before usin# the d



SAP $ata Services

• In this section5

Creatin# batch &obs and loadin# from flat files to *A+A

%uildin# transforms and usin# functions

Creatin# table &oins and utili-in# data mer#in#



uery 'ransformations

• 'ransforms are built/in ob&ects that chan#e

source data to the desired output

• 'he most commonly used transform is

Buery 'ransformB

• uery 'ransform enables you to5

:ilter and select data from a source

7oin data from multiple sources

ap columns from input to output schemas

Perform data nestin# and unnestin#

Add ne( columns to the output schema

Assi#n primary eys to output schema



$4



Addin# a $ata :lo( 4b&ect to the Worspace

!. $ra# a data flo( icon from the tool palette to

the (orspace

2. $ouble clic on the data flo( to enter its

(orspace

When creatin# a reusable ob&ect, such as a data flo( ob&ect, it (ill

automatically appear in the local ob&ect library

'he tool palette contains icons (hich allo( the creation

of ne( ob&ects in the (orspace



Addin# a $ata Source to the Worspace

!. $ra# a data source Di.e., flat fileEfrom the local ob&ect library on

to the (orspace

2. Create a connection bet(een thedata source and 6uery



uery ditor 4vervie(

• 'he 6uery editor is a #raphical interface for carryin# out 6uery o

•

It contains three areas5

>. $ouble/clic on the

6uery transform to open the

;uery ditor<

Parameters area

ScheSchema in area



Settin# p the 4utput 'able

?. $ra# the desired output fields to ;Schema 4ut<

from the ;Schema In< section

It is not necessary to dra# all fields from schema in to schema

out unless you (ant all the fields to appear in schema out



Creatin# a +e( 4utput Column

1. Right-click on an outut field a

select !Ne" #utut $olu%n&

'. Select "here to insert the ne"

colu%n

+e( columns can be created to display results from calculations



$efinin# Column Properties

>. 'he ;Column Properties< (ill pop

up for you to define and rename

the ne( column and its properties

Five the column a descriptive name that properly

identifies (hat the column is used for



sin# :unctions to Load $ata to *A+A

!. $o

>. Select the ap

then the spec

:or this demo, (e (ant to calculate the number of days a

case (as open

2. Clic on ;:unctions<



Settin# p the :unction

?. $efine the input

parameters for the function

se the drop

state the inpto avoid typo

+otice the updated code in this panel for the +4G$A=SGCASG4P+ column a

definin# the input parameters. 'his formula (ill deliver the number of days fro

4$A' to C$A', #ivin# us a measurement of ho( lon# it taes to close a cas



Addin# an 4utput 'able to the Worspace

!. $ra# and drop a table template in the

(orspace to be our output table

A template ta

the specific (ithout th

source dat

populated in

based on

re6uirements

2. Lin the 6uery to the template table

A template table is an o

used as a tar#et for data (hen a &ob #ets e)ecute

can also be saved in the

use as a data source at



)ecutin# a 7ob

!. 9i#ht/clic on the &ob and select e)ecute

'o analy-e any issues that may occur durin# data loadin#, clic ;

auditin#< and mae sure that ;se collected statistics< is chec



7ob Lo# 4vervie(

• 'he lo# file displays a list of actions in the &ob e)ecution. If any err

error icon (ill appear. 4ther(ise, ;7ob is completed successfully<

• 'he &ob lo# has five columns5

Pid5 'id5

+umber5

'ime Stamp5

essa#e5

Process thread identification number of the e)ecutin# thread 'hrenumber of the thread

+umber prefi) of the error follo(ed by a number

$ate and time the thread #enerated a messa#e

rror description of the thread



7ob Lo# 4vervie( Dcont.E

A &ob (ith errors (ill sho( the error icon

$ouble/clic on the er

of errors as



*o( to Previe( the 4utput 'able

!. Clic on the $ata :lo( to open its (orspace

2. Clic on the ma#nify #lass of the output ta

vie( data in the output table

+otice that the column created earlier is formatted correctly

number and that the data is the result of the function defined



SAP $ata Services


Creatin# batch &obs and loadin# from flat files to *A+A

%uildin# transforms and usin# functions

Creatin# table &oins and utili-in# data mer#in#



(se the )uer* +ransfor% ,R# clause to

oin the t"o sources/ )uer* and 0oin

7oinin# $ata from '(o Sources

• Source ! has the Car Case $escription, (hile Source 2 has the Solution to the case

• 'he 6uery transform (ill combine the data from the t(o sources to produce a resu

overall case solution

Source 1

Source '



9esult from a 'able 7oin

!. 4nce the tables have been &oined in the 6uery

transform, e)ecute the &ob as discussed earlier

+otice in the output table ho( the Solutio

source is combined (ith the fields from

2. nter the d

clic on the

the results



er#es 4vervie(

=ou can mer#e ro(s from t(o or more sources into a sin#le data set

All sources must have the same schema to e)ecute the er#e 'ransform

the same number of columns, same column names and the same da



*o( to Create a er#e

!. 'o mer#e t(o sources, add a 6

source to format all the data to

both sources

2. 7oin the 6ueries to a ;er#e 'ransform<

>. When openin# the ;er#e 'ransform,< notice ho(

fields and data types match for all output and input f



Avoidin# $uplicated $ata in er#es

?. 'o avoid duplicate ro(s, add a 6uery

transform to display distinct ro(s only

". )ecute the &ob to complete the mer#ed tab



What We1ll Cover


• Levera#in# %W )tractors in *A+A, %W and in $ata Services

• Wrap/up



Levera#in# %W )tractors


$ata Load Processes for %W on SAP *A+A

Fuidelines for )tractor Selection %uildin# $ataflo(s (ith )tractors

Loadin# $ata from non/SAP Sources to SAP *A+A




• %W on *A+A supports all e)istin# %W H.? 'L processes5

• Standard %W %usiness Suite )tractors

SAPI

SAP %usiness4b&ects $ata Services ?.)

:lat :iles DCS@, LS, LSE

$% Connect

$ Connect

Web Services

If you use SL' DSAP L' 9eplication ServerE for direct

loadin# into *A+A, can can create tables autimatically



SL' and $C ethods for Loadin# $ata to *A+A

• $ata that is replicated real/time directly to *A+A in SL' can be used by %W

SL' real/time custom data marts can be combined from the *A+A schema (ith %W data models InfoProviders, 4pen4$S @ie(s and nhanced CompositeProviders in %W H.?

• 'he $irect )tractor Connection D$CE can be used to pull data from the %usiness Suite d

by usin# Standard %W )tractors

$ataSource )tractors in the SAP %usiness Suite systems can load data directly from the %W Inf

It also (ors (ith #eneric $ataSources and (ith any $ataSources (ith customer e)tensions

It is available since *A+A SP/? and can be implemented usin# a sidecar approach usin# the %W

CC system, so you can levera#e e)istin# foundational data models from %W business content i

mart

%W e)tractors offer business lo#ic and semantics to the source

data, but SL' only replicates tables on a !5! basis



$C / $ata Load Processes for %W on SAP *A+A

• $C is meant to levera#e %W business content $ataSources to minimi-e comple)ity of da

(ithin SAP *A+A data marts and to accelerate implementation pro&ects

ore information about $C can be found in SAP +ote !KK"K2



+e( %usiness Content in %W H.? for *A+A

• SAP %usiness Content e)tractors

4ri#inally developed for use (ith %W

Semantic depiction of 9P data for analytic uses

+o interruption (hen concurrently used for %W e)traction Can be implemented to populate other data marts, instead of, or

in parallel, (ith %W loads

Some )amples5

%W/%C'/CA Cros

%W/%C'/%%P %usProcurement

%W/%C'/C4 Cont

%W/%C'/$4C $o

%W/%C'/S$ Sales

%W/%C'/L4 Lo#is

%W/%C'/F+ Fen

%W/%C'/'C' 'ec

Cocpit

%W/%C'/P= Payro

%W/%C'/IS9 9eta



%4$S for %W on SAP *A+A

%4$S source systems enable users to create $ataSources that precisely mirror the structure

)ternal System sources also allo( users to create $ataSources (ith arbitrary structures tha

any %4$S &ob

%W to %4$S connectivity5



%4$S Loads for %W on SAP *A+A

• When usin# an SAP data source in %4$S, the $ataStore must be defined as a SAP connec

• 'o import the metadata into the $ataStore. %4$S offers these %W options

$ata reads from an InfoProvider, such as a $ataStore ob&ect, InfoCube, or Info4b&ect5

SAP 4pen *ub $estination service 0 performs a %W process that reads data from thand then loads an 4pen *ub destination table that %4$S reads as a source

Automatically #enerated A%AP pro#rams or 9:C functions that read data from fund

the $ataStore ob&ect, InfoCube, or Info4b&ect

'he old Application Lin nablement

replaced (ith synchronous 9emote :

i h



%4$S (ith %W H.? on SAP *A+A

• $ata loads into a %W $ataSource or InfoSource usin# a %4$S

batch &ob (hich can be initiated by5

)ecute the %4$S batch &ob that automatically prompts a

%W InfoPaca#e to load an InfoProvider

Implement an InfoPaca#e (ithin %W to schedule and

e)ecute the %4$S load &ob (hich has been e)ported to %W

• sin# standard e)tractors from 9P, you can use the

4perational $elta ueue D4$E to move data to one or more

%W systems

• An 4$ inside %W can also be used to move data to other

%W systems

%4$S can also load data into a $ataSource in the Persistent Sta#in#

Area usin# the sta#in# %API (ith support from the 9:C server.

9:C C i



9:C Connections

• 'o load data from %W to *A+A usin# %4$S, an 9:C connection must be e

• In order to e)tract data from a %W InfoCube, a %W 4pen *ub and a %W process chain are re6uired5

!. Create an 9:C $est D'CPJIPE as a re#istered pro#ram in %W

2. :orm a connection to the SAP 9:C server usin# the $ata Services ana#ement Console (hich (

9:C connection that (as established

>. $evelop a %4$S &ob to run the %W 4pen *ub (hich (ill e)tract the cube data to the %W $% tabl

?. %4$S then re6uests the data via 9:C throu#h the %4$S server (hich (ill load it into *A+A

". SAP *A+A Studio inte#rates (ith %4$S to access e)ternal metadata

L i %W t t








F id li f t t S l ti



Fuidelines for )tractor Selection

!. An e)tractor may be replaced by a ne(er version so use

the latest version

2. If possible, use e)tractors that have detailed data and not

the ones that already a##re#ated the data

>. %e a(are of primary ey information in e)tractors as this is

not used by %W and therefore may be missin# or erroneous

?. $o not assume that all e)tractors of a delta process have

same field values

". Carefully analy-e re6uirements for each e)tractor

K. ach e)tractor has particular capabilities, especially as it relates to delta processes

H. +umerous e)tractors return a handful of data and delta Dtracin# chan#esE does not mae sense for t

. 4ther e)tractors share the records that have chan#ed but do not return deletions, and no deletions a

source

M. 'here are also e)tractors that provide complete information from insert, updates, before and after ima

deletes













%uildin# $ataflo(s (ith )tractors




• sin# e)tractors is similar to usin# tables5

%oth are 6ueried and return data upon e)ecution

'he primary difference is that unlie table readin#, it is recommended toe)tractor to a re#ular dataflo(

When an SAP table is added to a re#ular dataflo( it is read via the 9:CG

function module (hich returns all the data in one function call

'his is acceptable for a fe( ro(s of data, but not for lar#er amounts of d

)tractors differ since the API is more lie a SL dataset 'he e)tractor is prepared and e)ecuted, and the arrays of records are se

retrieved and the data is immediately loaded to the do(nstream transfor

'he initiali-ation fla# is used to mar the e)tractor as

readin# all of the data, and not &ust the delta.

%uildin# $elta Loads and $ataflo(s (ith %W H ) )tractors



%uildin# $elta Loads and $ataflo(s (ith %W H.) )tractors

• It is the same concept as the truncate table fla# used in the tar#et table for

Ddurin# the first run the tar#et table is empty so truncation is unnecessaryE

•

If the initial load is re/e)ecuted, the dataflo( ensures that the data is truncaapplies for the e)tractor, as (ellE

• If the read never taes place, then settin# the initiali-ation fla# is not impor

• If the initial load is re/run and the initiali-ation fla# is set to ;+o<, then only

(ould be retrieved





• A #lobal variable can be used if there are

t(o dataflo(s5

4ne (ith the initiali-ation fla# set to

;=es< and the other to ;+o<

• In most cases there (ill be t(o dataflo(s

since the initial load includes the data as

6uicly as possible and the delta load uses

its lo#ic to account for modifications and

deletes• As of %W H.?, you can no( activate ob&ects

inside the dataflo( (ithout leavin# the 'L

modelin# screen

'he e)tractor date/time is used to associate an e)tractor run (ith a d

is not needed for the initial load, this date (ill help (ith buildin# de









+on/SAP Sources to SAP *A+A usin# SL'



+on/SAP Sources to SAP *A+A usin# SL'

• A database connection is necessary to connect

from a non/SAP source to SL' DSAP Landscape

'ransformationE

• 'ables can be loaded but they cannot be

automatically replicated, meanin# the addition of

ne( tables or columns after the initial load must be

done manually

• ach table must have a primary ey or inde) (ith

basic data types• 'he read modules are located in the SAP L'

9eplication Server for non/SAP sources

• 'he user roles that are defined (hen (orin# (ith

non/SAP sources depend on the source system

+on/SAP Sources to SAP *A+A (ith SL'



+on SAP Sources to SAP *A+A (ith SL'

• 'o manually e)ecute replication and perform the initial load on a non/

SAP source for a certain table, the database user must have permission

to5

Selectin# from a table, creatin# a table in the #iven schema Dto#enerate the lo##in# tableE

Selectin# from and deletin# the lo##in# table

Creatin# and deletin# database tri##ers

Creatin# and deletin# synonyms for the table

•

4nce the initial load process is complete, any chan#es that are made inthe non/SAP source system to the *A+A system are reflected by creatin#

a local report entitled NGC9A'G$$G'A%LS in the SL' system

• When this report is e)ecuted, the $$2L and $$2' SAP dictionary

tables are automatically updated

What We1ll Cover



What We ll Cover


• Levera#in# %W )tractors in *A+A, %W and in $ata Services

• Wrap/up

Where to :ind ore Information



Where to :ind ore Information

• https5JJ(((.sap/press.comJsap/hanaG>KHJ

%&arne %er# and Penny Silvia, SAP HANA: An Introduction DSAP P9SS,

• (((.saphana.comJ(elcome SAP1s main pa#e for all SAP *A+A/related information

• http5JJhana.sap.comJtry.html

'ry *A+A for free

• http5JJscn.sap.comJcommunityJhana/in/memory

SAP *A+A and In/emory Computin# by SAP *A+A Community

H Oey Points to 'ae *ome



H Oey Points to 'ae *ome

• SAP $ata Services can transform, refine and deliver data for the $W

• ultiple data sources can be used for $ata Services, includin# :lat :iles, $

Schemas, )cel Worboos, and more• se built/in transforms, (hich are ob&ects that process source data to chan

• SAP *A+A is not &ust %W and 9P, but can inde) data from a variety of sou

• $C is meant to levera#e standard %W business content $ataSources as a

minimi-in# comple)ity of data modelin# tass (ithin SAP *A+A data marts

accelerate timelines for implementation pro&ects

• $C can be used to pull data from the %usiness Suite directly to *A+A by

Standard )tractors

• 'o load data that is e)tracted from a %W system into *A+A usin# %4$S, an

connection must be established, but the tass are very simple

=our 'urn



*o( to contact me5

$r. %&arne %er#

bber#Qcomerit.com

Please remember to complete your session evaluation

$isclaimer



SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate

countries. All other product and service names mentioned are the trademarks of their respective companies. Wellesley nformation Services is neither owned nor cont

Date post:	05-Mar-2016
Category:	Documents
Upload:	mrudul007
View:	29 times
Download:	0 times

Guidelines ETL HANA v3 (1)

Documents