Semantic Data Modeling: The Key to Re-usable Data

Stephen Brobst

Chief Technology Officer

Teradata Corporation

stephen.brobst@teradata.com

617-422-0800

2 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Not just a collection

of subjects...

Activity Party Product Account

Single, Integrated System

...but also their

relationships

Party Product

Account Activity

Dont model subjects individually!

Model your entire

business!

Enterprise Information Management Data Modeling

3 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Functional Views

Sales Marketing Finance Rates/

Regulatory

Customer

Service Risk

Demographics Pricing

General Ledger

Promotions

Products Safety Engineering

Production

HR

Contracts

Works OK for OLTP, but causes

data chaos for BI applications.

4 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Business Intelligence Requires Data Integration

Product Data

Customer Data

Account Data

Transaction Data

G/L Data

Market Data

External Data

5 Copyright 2013. Stephen Brobst. Do not duplicate without written permission Copyright 2005, Stephen A. Brobst. All rights reserved.

Data Modeling Techniques

Key observation: Practitioners in the data warehousing industry frequently confuse construction of the semantic data model, logical data model, and physical data model.

A semantic data model (SDM) captures the business view of information for a specific knowledge worker community or analytic application.

A logical data model (LDM) captures the business relationships in the enterprise information independent of a specific analytic application or departmental view.

A physical data model (PDM) captures the implementation design of tables in the data warehouse.

6 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Data Model Deployment

Conceptual Data Model

Project A Project B Project C

Enterprise Data Standards

Subject Area A

Enterprise Logical Data Model(3NF)

xxxxxxx

xxxxxxxxxx

xxxxx

xxxxxxx

xxxxxxx

xxxxxxxxxx

xxxxx

xxxxxxx

xxxxxxx

xxxxxxxxxx

xxxxx

xxxxxxx

xxxxxxx

xxxxxxxxxx

xxxxx

xxxxxxx

xxxxxxx

xxxxxxxxxx

xxxxx

xxxxxxx

xxxxxxx

xxxxx

xx

Subject Area

A

Physical Model Realization

Design Meta

Data

Semantic Model Views

Subject Area

B

Subject Area

C

Single Physical Data Model

Subject Area B

7 Copyright 2013. Stephen Brobst. Do not duplicate without written permission Copyright 2005, Stephen A. Brobst. All rights reserved.

Semantic Data Modeling

Semantic data modeling is a logical data modeling technique; the semantic view of information does not necessarily need to be physicalized in the database.

There may be a different semantic data model for each department/applications that uses the data warehouse.

Dimensional modeling is a common technique for constructing the semantic data model for an analytic application, but is not the only viable approach.

8 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Dimensional

Physical Data Extensions

Different Semantic Model Designs are Appropriate for Different Types of Knowledge Workers

Normalized Generic Structures

Index choices & selective table denormalizations

Relational ADS Application

9 Copyright 2013. Stephen Brobst. Do not duplicate without written permission Copyright 2005, Stephen A. Brobst. All rights reserved.

Physical Data Model

Physical data model represents the tables constructed in the database.

Recommendations:

Use the (3NF) LDM as the starting point for the PDM with selective denormalization when appropriate for (primarily) performance reasons.

Overlay (dimensional) SDM on top of PDM using views and/or semantic metadata in your BI tool.

Design LDM first, then use application-specific business requirements to derive the SDMs and performance considerations to map into the PDM.

10 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Semantic Models Should be BI Tool Agnostic

MicroStrategy

Teradata OLAP Connector

Tableau

Tier 3 Access

Tier 2 Integrated

Tier 1 Acquisition

11 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

A collection of data modeling assets that help make database design and development faster and easier for the access layer:

> Access layer provides path for data from the integrated data model to end user consumption.

> When this layer not well-designed, it can impact speed, security, and simplicity in developing and delivering reports, BI applications.

Re-usable building blocks provide flexibility and consistency to the development process:

> SMBBs include pre-built semantic models.

Focuses on a specific analytic need in a specific industry:

> For example, Communications Mobile Revenue Analytics.

SMBBs are to the semantic layer as iLDMs are to the integrated layer of a data warehouse implementation.

What is a Semantic Modeling Building Block (SMBB) Portfolio?

12 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Dimensional Model

Dimension Building Blocks

Dimension Building Blocks Support a Range of Analytical Needs

Fixed, Normalized Hierarchy Fixed, Flattened Hierarchy Variable Depth Hierarchy (Recursive)

13 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

What are SMBBs? How are they related to an LDM?

Building from the Foundation for your Data Warehouse:

An LDM is like a blueprint for a house that you are building. It serves as the foundation for your integrated data warehouse.

The SMBBs are like room designs that meet specific homeowner needs. Different rooms need different designs based on their purpose. Similarly, for each new business application, new semantic models are needed.

SMBBs provide different designs (building blocks) for the modeler to choose from in building the semantic models.

These flexible, reusable building blocks can be used for other analytic needs.

14 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Q: Where does it all start? A: Business requirements drive the process!

Relationships between the Three Types of Data Models

The Logical Model is

used to drive

generalization and

support source data

leverage and reuse.

Logical Data Model Physical Data Model Semantic Data Models

Data access patterns

Support data re-use

The Semantic Model

captures data

access patterns that

must be supported

by the core physical

model.

The Physical Model

provides core

support for data

integration within

the information

architecture.

15 Copyright 2013. Stephen Brobst. Do not duplicate without written permission

Semantic Layer Benefits

Efficient table joins can be encouraged inside the SDM views.

Views are low maintenance objects.

Views do not consume database space.

Join indexes (JIs) and aggregate join indexes (AJIs) can be created based on the access paths embedded in the SDMs.

PDM is not compromised with new application requirements.

Protection of code assets.

16 Copyright 2013. Stephen Brobst. Do not duplicate without written permission Copyright 2005, Stephen A. Brobst. All rights reserved.

Conclusions

Critical to distinguish between logical data modeling, semantic data modeling, and physical data modeling.

Separate the implementation of the semantic model from the physical data model (PDM) deployment for maximum flexibility.

Selective use of PDM extensions to optimize performance.

Either ANSI standard views of the semantic metadata within your BI tool of choice can be used for creating a semantic data layer without sacrificing flexibility of the PDM.