Building Agile Data Warehouses with Ralph Hughes

1 March 22, 2012

© Kalido I Kalido Confidential I March 22, 2012

Building Agile Data Warehouses

Kalido Lessons From Leaders Webcast Series

Featuring Ralph Hughes

2 March 22, 2012


Logistics

Attendees will be on mute for the call

Type your questions into the Questions box

Webcast is being recorded and will be available for replay

Request a copy of today’s slides by sending an email to: [email protected]

Join the conversation online by using the #Kalido hashtag. Follow us @kalido

mailto:[email protected]

3 March 22, 2012


Today’s Speakers

Ralph Hughes Chief Systems Architect, Ceregenics

John Evans Director of Product Marketing, Kalido

4 March 22, 2012


Discussion Topics

Waterfall vs. Agile Methods

Benefits of an Agile Approach

How to Implement Agile Methods with Agile Tools

Q&A

BUILDING AGILE DATA WAREHOUSES

Ralph Hughes, MA, PMP, CSM

Chief Systems Architect

Slide 6 Copyright © 2012, Ceregenics. www.ceregenics.com

WATERFALL AND PROJECT RESOURCES

Single Pass

S/w Reqs System Reqs Design Testing Operations Coding Analysis


AGILE DW/BI IN A NUTSHELL

The “Project Backlog” Prioritized for value and dependencies

6 to 10 developers

+ iterative method

A growing

“release pool”

Promote to

production


WATERFALL HANDOFF TAKE FOREVER

Project Architects Data Modelers ETL Designers

Waterfall Project Start-Up Time

80% of the value

last 20%

80% of the value

last 20%

80% of the value

last 20%


AGILE IS RECURSIVE 80/20

80%

100% 100% 100% 100% 100%

Agile

Waterfall

Time & Cost Savings

Requirements

Vision

Data Model

Specs

Coding Collaboration between

specialties so that one

person is always within

their comfort zone...this

is where agile gets much

of its speed


INMON/KIMBALL PARALYSIS

EDW Architect Project

Architect

Star

Sources

3NF


INMON/KIMBALL PARALYSIS

Sources

3NF

Star

Release 1 Compromise

EDW Architect Project

Architect


STARTING POINT: 3NF

Starts with 7 tables


STEP 1: IDENTIFY PIVOTAL BUSINESS KEYS


STEP 2: TRANSITION TO “LINKS”


STEP 3: SEPARATE OUT CONTEXT

From 7 to 16 tables


STEP 4: EXTEND FOR NEW REQUIREMENTS

Note: No impact upon

rest of model!

“Three-Way Robust”

build in little pieces

re-engineer with

small scope

scales linearly


EXAMPLE: SPONSORED IMMIGRATION DM

Project Sponsor:

“Can’t DWBI focus first

on just the key integration points,

get some basic counts, and

then go back to add in

all the complex attributes?”


AGILE BI FOR SPONSORED IMMIGRATION


APP GENERATORS LET YOU SKIP STEPS

Logical

Data Model

Physical

Data Model

Integration

Layer

Dimensional

Layer

Semantic

Layer

BI

Layer

Integration-Layer Generator

Traditional

90%


MODEL-DRIVEN DEV: ASSOCIATIVE ENGINES

Link

Types

Thing

Types

Logical Business Model

Physical Data

Objects & ETL

Sem

Int

Star

Ge

ne

rate

s


APP GENERATORS LET YOU SKIP STEPS

Logical

Data Model

Physical

Data Model

Integration

Layer

Dimensional

Layer

Semantic

Layer

BI

Layer

Model-Driven Application Generator

Integration-Layer Generator

Traditional

90%

90%


WATERFALL VS. BUSINESS-IT ALIGNMENT

Business

IT

“Digest Data”

Time

Build Big Fix

“Digest”

Customer

Touch

Point

Customer

Touch

Point


AGILE’S IMPACT ON THE ENTERPRISE “Digest Data”

First release

Traditional

Phase 2

Value of

Agile Warehousing

Time

Customer Touch Points

Business

IT

24 March 22, 2012


Traditional Approach Extends Time To Decision Making

Time to Deliver

Business Value

Traditional

Adapted from “The Time Value Curve” by Dr. Richard Hackathorn, Bolder Technology Inc.

25 March 22, 2012


Evolution Of Tools And Methods

Need to acknowledge that things change

Architecture that enables integration and incremental builds

Optimized and automated processes for optimal performance

End-to-end governance to ensure semantic consistency and quality

Formal agile methodology to deliver rapidly and support change

Extract maximum value from each step in the process

Maintain connection to the business

Agility Requires a New Approach and Supporting Technology

26 March 22, 2012


Centrally Capture Requirements

27 March 22, 2012


Model-Driven Automation

Operations

Process Automation Task Execution & Monitoring

Deployment and Migration Archiving

Restore for Model and Data Undo Loads

Audit and Logging Job Definition with Dependency

BI Delivery

Metadata Management for BOBJ

Native XLS Pivot Table Generation

Native QlikView Generation

Metadata Management for MSAS

Metadata Management for COGN

Report-Time Formula Management

Testing

Built-in Integrity Checking

Aggregate Task Results

Excel Integration for User reconciliation

Data rollback and batch reload for system test

User Interface for data browsing & troubleshooting

Release to Production

Version Management

Object level Change Management

Model Migration

Generic Export/ Import for Data Migration

Object Level Dependency for Migration Versions

Model Comparison Report

Business Information Model Driven Automation

Data Integration

Data Validation

Suspense and Exception Handling

Data Sourcing and Field Mapping

Delta Detection

System Key Management

Code Management and Lookup

Currency and Units of Measure

Contra Processing

Data Export & Purging Post Processing Housekeeping

Modeling

Star and Snowflake Schema

Physical Schema Management

Slowly Changing Dimensions

Data Mart and Aggregates

Data Load and Index Management

Rollup Path Awareness

Incremental Summary Generation

Convert Existing Logical Models

Name & Label Management

28 March 22, 2012


Iterative Agile Approach

Configure & Load

Test & Validate

Build Phase

Define Phase

• Adaptive Project Management

• Version Control

Detail Requirements & Design

Iterate Phase

29 March 22, 2012


Shorten The Cycle, Maximize Business Value

Time to Deliver

Time to Value Benefit

Business Value

Business Value Benefit

Kalido

Traditional

Adapted from “The Time Value Curve” by Dr. Richard Hackathorn, Bolder Technology Inc.

30 March 22, 2012


Architecture That Supports Incremental Build

Data Mart 1

Expand rather than rebuild

31 March 22, 2012



Data Mart 1

Data Mart n

Expand rather than rebuild

Leverage existing value and build incrementally

32 March 22, 2012



Data Mart 1

Data Mart n

New Requirement Expand rather than rebuild

Leverage existing value and build incrementally

Pay as you grow rather than huge upfront capital expenditures

33 March 22, 2012


Material Valuation

AR

Balance Sheet

Bill of Material Component Cost

Customer Contacts

Demand Planning

Financial Plan

Income Statement

Plant Level Delivery

Profitability Analysis

Repack Activity

SAP Inventory Adjustments

Stock Transfer Orders

Vendor Inventory

Bill of Material

Inventory Age

POC Execution Statistics

Revenue Recognition Sales

Sales Visit Target

Shelf Life

Variable Sales Compensation

Account Expenses

Account Statistics

Price Account Mappings

Telesales Inventory

Telesales Inventory Strike Rate

Truck Loads

Daily Inventory Reporting (LCBO)

Territory Supplementary Expenses

Consumer Price Survey Management

Inventory Reporting (LCBO)

POC Survey Distribution

Trade Spend Bookings

Liquor Control Board Product Mapping

Trip Management

US Distribution

US Movements

US Product Margin

US Production Plan

US Telesales Targets

Trade Spend

Wholesaler Profitability

CRM Sales Plans

US Sales Visit Management

Customer Survey Management POC Surveys

IT Services Actuals US Account Manager Targets

IT Services Budget US Retail Account Management Targets

IT Services Targets US Retail Sales

US Wholesaler Profitability US Sales

Workers Compensation Management

Telesales Plan

Market Share

Sales Visit Management

Telesales Targets

Key Account Sales Targets

Price Variance Reporting

Sales Orders

Daily Inventory

WIP Job Production

ABC

HR Accident Management

External Distributor Product Substitution

Production Complexity

Warehouse Productivity

Commercial Sales Targets Production Schedules

In Transit Inventory Production Yield

Line Speeds Product Level Shipments

PDA Expenses Split-Week Demand Plan

Production Storage Targets

Production FTE Targets Target Production Loss

Production FTE Utilization Utility Usage & Targets

Production Loss

Distribution Targets

Product Repackaging

Sales Targets

Standard Pricing

Channel Distribution

Volatile Sales Planning

Product Account Distribution

Reseller Sales

Account Product Distribution

Brand Costs Product Distribution

Brand Profitability Product Margin

Cost of Sales Product Substitution

Daily Sales Sales Actuals

Financial Balances Sales Expenses

FX Sales Plan

Key Account Mgt Brand Level Shipments

Pricing Weekly/ Monthly Demand Plan

2004Q2 2004Q3 2004Q4 2005Q1 2005Q2 2005Q3 2005Q4 2006Q1 2006Q2 2006Q3 2006Q4 2007Q1 2007Q2 2007Q3 2007Q4 2008Q1 2008Q2 2008Q3 2008Q4 2009Q2 2010Q1 2010Q2 2010Q3

Warehouse Expansion By Subject Area

Time Progression by Quarter

Bu

sin

ess M

etr

ics C

ov

ere

d

34 March 22, 2012


The Kalido Solution

Software automation for data warehousing that runs in database and eliminates lots of labor that is typically required to design, build, and change the analytic foundation

Data Infrastructure

Logical Modeling

Physical DBMS Design

ET

L

BI S

em

an

tic

s

Data Sources

Data

Warehouse

Mart

Mart

Mart

Legacy

Governance and Stewardship

SCM

CRM

External

ERP

Finance

Business Intelligence

Customer Analytics

Sales Analytics

Financial analytics

Margin Analysis

Executive Dashboard

Regulatory Reporting

Automation

35 March 22, 2012


Agility To The Extreme

Antiquated big-bang methods and siloed approaches are giving way to well-managed Agile methodologies

Agile methods provide short term value without sacrificing long term goals

Agile tools to automate the mechanics of modeling, design, data integration, data access, test and release to production reduce risk and shorten time to value

Pay-as-you-grow smoothes the cost curve and allows your budgets to stay in sync with derived value

36 March 22, 2012


Next Steps

Attendees will receive our whitepaper on “Ensuring Agility in Your Data Warehouse” and a free chapter from Ralph’s book

Download Kalido Business Information Modeler http://www.kalido.com/business-modeling-community.htm

Take the Kalido Challenge – http://info.kalido.com/kalido_challenge.html

Read our blog about Kalido Information Engine http://blog.kalido.com/category/information-engine/

Contact us! +1.781.202.3200, press 1

http://www.kalido.com/business-modeling-community.htm






http://info.kalido.com/kalido_challenge.html

http://blog.kalido.com/category/information-engine/



37 March 22, 2012


Thank you for attending!

Date post:	19-Oct-2014
Category:	Technology
View:	2,729 times
Download:	1 times