Date post: | 03-Mar-2017 |
Category: |
Technology |
Upload: | dataversity |
View: | 371 times |
Download: | 2 times |
Data Modeling & Business IntelligenceDonna Burbank
Global Data Strategy Ltd.
Lessons in Data Modeling DATAVERSITY Series
February 23rd, 2017
Global Data Strategy, Ltd. 2017
Donna Burbank
Donna is a recognised industry expert in information management with over 20 years of experience in data strategy, information management, data modeling, metadata management, and enterprise architecture. Her background is multi-faceted across consulting, product development, product management, brand strategy, marketing, and business leadership.
She is currently the Managing Director at Global Data Strategy, Ltd., an international information management consulting
company that specialises in the alignment of business drivers with data-centric technology. In past roles, she has served in key brand strategy and product management roles at CA Technologies and Embarcadero Technologies for several of the leading data management products in the market.
As an active contributor to the data management community, she is a long time DAMA International member and is the Past President & Advisor to the DAMA Rocky Mountain chapter. She was also on the review committee for the Object Management Group’s Information Management Metamodel (IMM) and a member of the OMG’s Finalization Taskforce for the Business Process Modeling Notation (BPMN).
She has worked with dozens of Fortune 500 companies worldwide in the
Americas, Europe, Asia, and Africa and speaks regularly at industry conferences. She has co-authored two books: Data Modeling for the Business and Data Modeling Made Simple with ERwin Data Modeler and is a regular contributor to industry publications. She can be reached [email protected] is based in Boulder, Colorado, USA.
2
Follow on Twitter @donnaburbankToday’s hashtag: #LessonsDM
Global Data Strategy, Ltd. 2017
Lessons in Data Modeling Series
• January 26th How Data Modeling Fits Into an Overall Enterprise Architecture
• February 23rd Data Modeling and Business Intelligence
• March Conceptual Data Modeling – How to Get the Attention of Business Users
• April The Evolving Role of the Data Architect – What does it mean for your Career?
• May Data Modeling & Metadata Management
• June Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling
• July Data Modeling & Metadata for Graph Databases
• August Data Modeling & Data Integration
• September Data Modeling & MDM
• October Agile & Data Modeling – How Can They Work Together?
• December Data Modeling, Data Quality & Data Governance
3
This Year’s Line Up
Global Data Strategy, Ltd. 2017
Agenda
• Where does Data Modeling fit with the rise of Self-Service BI?
• How Data Modeling is the “Intelligence Behind Business Intelligence”
• Creating business meaning & context
• Understand source and target data systems
• Optimize data structures to align queries with reports
• Integration with other tools (BI, ETL, etc.)
• Summary
4
What we’ll cover today
Global Data Strategy, Ltd. 2017
The Importance of Data Modeling in Business Intelligence
5
If You Can Read This, Thank a Data Modeler!
Global Data Strategy, Ltd. 2017
The Rise of the Data-Driven Business
Data, more than ever, is seen as a key business asset and strategic differentiator.
6
Global Data Strategy, Ltd. 2017
The Rise of Self-Service Business Intelligence
• As a result of this growing importance of data, the interest in self-service data reporting has increased among data-savvy business users.
• The availability of tools & data sets has made it easier for business people to do their own data manipulation & reporting
• Self Service BI & Data Manipulation – the tools are slick!
• Accessible Data & Open Data Sets – the amount of data available is amazing!
• Tech-Savvy Business Users – this isn’t any harder than a spreadsheet!
• While this offers great opportunities, it can also be fraught with challenges.
• Data modelers and the models & metadata they create can make the job of business intelligence easier for both BI professionals and the casual BI reporting user.
7
Global Data Strategy, Ltd. 2017
Users are Often Frustrated with Self-Service BI
8
What they want
What they often get
Global Data Strategy, Ltd. 2017
Data is Only as Good as the Metadata
9
Open Data Example: Road Safety - Vehicles by Make and Model
Global Data Strategy, Ltd. 2017
Metadata Matters
With Self-Service BI and Analytics, attention needs to be paid to the quality, context, & structure of data
Raw data used in Self-Service Analytics and BI environments is often so poor that many data scientists and BI professionals
spend an estimated 50 – 90% of their time cleaning and reformatting data to make it fit for purpose.(4
Source: DataCenterJournal.com
Correcting poor data quality is a Data Scientist’s least favorite task, consuming on average 80% of their working day
Source: Forbes 2016
(aka Data Models & Metadata)
Global Data Strategy, Ltd. 2017
Business Intelligence is a Key Driver for Metadata Management
11
Global Data Strategy, Ltd. 2017
Data Modeling for Data Warehousing & Business Intelligence
• What is the definition of customer?• Where is the data stored?• How is it structured?• Who uses or owns the data?
Data Warehouse BI Report:Customers by Region
• What are the definitions of key business terms?• What do I want to report on?• How do I optimize the database for these reports?
Data Modeling helps answer:
For Data Warehousing For BI Reporting
Data Modeling helps answer:
• Data Modeling is the “Intelligence behind Business Intelligence”• Creating business meaning & context
• Understand source and target data systems
• Optimize data structures to align queries with reports
Show me all customers by region
Source Systems
Relational ModelDimensional Model
Global Data Strategy, Ltd. 2017
Finding Balance – Model What Matters
13
• It’s important to find a balance between • Managing & modeling “trusted data sets”• Giving users the flexibility to explore.
• Most users will find these trusted data sets a welcome asset, but don’t want to be restricted from doing data exploration when appropriate.
IoT
Log Files
Data Warehouse
Master Data
Reference Data
Structure Flexibility & Exploration
Global Data Strategy, Ltd. 2017
Data Models Levels – Both Business & Technical
14
Conceptual
Logical
Physical
Purpose
Communication & Definition of Business Terms & Rules
Clarification & Detail of Business Rules &
Data Structures
TechnicalImplementation on a Physical Database
Audience
Business Stakeholders
Data ArchitectureBusiness Analysts
DBAsDevelopers
Business Concepts
Data Entities
Physical Tables
• For Data Modeling for Business Intelligence, it’s important to focus on both the business & technical views.
Global Data Strategy, Ltd. 2017
Business Meaning & Context is Critical
15
Show me all customers by region
Businessperson Data Architect
“Does this include current customers only? Or lapsed customers as well?
“Do we have to obfuscate PII?”
Global Data Strategy, Ltd. 2017
The Importance of Business Definitions
From Data Modeling for the Business by Hoberman, Burbank, Bradley, Technics Publications, 2009
Global Data Strategy, Ltd. 2017
Conceptual Data Model
• Communication & definition of core data concepts & their definitions
Global Data Strategy, Ltd. 2017
Data Model Metadata Can Be Used by Many Roles
18
Business Person(e.g. Finance)
What’s the definition of “Regional Sales”
Auditor
How was “Total Sales” calculated? Show me the
lineage.
Data Architect
What is the approved data structure for storing customer
data?
Data Warehouse
Architect
What are the source-to-target mappings for the DW?
Business Person(e.g. HR)
How can I get new staff up-to-speed on our company’s business terminology?
Global Data Strategy, Ltd. 2017
Data Model Design Layer Relationships
• Data model design layer mappings show the relationship between business terms and their physical implementations on a database platform.
19
Showing Semantic Mapping
Conceptual
Logical
Physical
Business Concepts
Data Entities
Physical Tables
Client
Customer
DB2TeradataOracle
CUST CUSTOMER CTABLE_16
In a Conceptual data model, there may be a concept called “Client” which is the term businesspeople use to describe the people they sell to and work with.
The Logical model might use the term “Customer” for that same concept.
Which may be implemented in a number of physical tables with varying naming conventions.
Conceptual
Logical
Physical
Business Concepts
Data Entities
Physical Tables
Global Data Strategy, Ltd. 2017
Data Modeling Creates an “Active Inventory” of Data Assets
• Know what data you have: Create a visual inventory of database systems
• Know what your data means: Communicate key business requirements between business and IT stakeholders
• Support data consistency: Build consistent database structures & support data governance initiatives
Sybase
MySQLOracle
Data Models
Teradata
Sybase
SQL Server
DB2
Teradata
SQL Server DB2
MySQLSQL Azure
SQL Azure
Oracle
Global Data Strategy, Ltd. 2017
Metadata Adds Context & Definition
• Metadata stored in data models provides valuable business & technical context.
21
Global Data Strategy, Ltd. 2017
Technical & Business Metadata
• Technical Metadata describes the structure, format, and rules for storing data
• Business Metadata describes the business definitions, rules, and context for data.
• Data represents actual instances (e.g. John Smith)
22
CREATE TABLE EMPLOYEE (
employee_id INTEGER NOT NULL,
department_id INTEGER NOT NULL,
employee_fname VARCHAR(50) NULL,
employee_lname VARCHAR(50) NULL,
employee_ssn CHAR(9) NULL);
CREATE TABLE CUSTOMER (
customer_id INTEGER NOT NULL,
customer_name VARCHAR(50) NULL,
customer_address VARCHAR(150) NULL,
customer_city VARCHAR(50) NULL,
customer_state CHAR(2) NULL,
customer_zip CHAR(9) NULL);
Technical Metadata
John Smith
Business Metadata
Data
Term Definition
EmployeeAn employee is an individual who currently works for the organization or who has been recently employed within the past 6 months.
Customer
A customer is a person or organization who has purchased from the organization within the past 2 years and has an active loyalty cardor maintenance contract.
Global Data Strategy, Ltd. 2017
Data Warehousing – An Example
• In the data warehouse example below, metadata for CUSTOMER exists in a number tools & data stores.
23
Sales Report
CUSTOMER
Database Table
CUST
Database Table
CUSTOMER
Database Table
CUSTOMER
Database Table
TBL_C1
Database Table
Business Glossary
ETL Tool ETL Tool
Physical Data Model
Physical Data Model
Logical Data Model
DimensionalData Model
BI Tool
Global Data Strategy, Ltd. 2017
Data Lineage
24
• Data Lineage shows the source to target mapping, or provenance for information.
• Many data modeling tools track this lineage through integration with ETL tools, or with internal mapping functionality.
Global Data Strategy, Ltd. 2017
Why Model the Data Warehouse
• Proper modeling of a data warehouse creates data sets that are:
• Easy to use
• Fast to access
• Combined with other data warehousing best practices around data integration, transformation, & governance, the data warehouse also creates data sets that:
• Contain high quality data
• Provide a broad, integrated set of data across the enterprise
25
Computing report…elapsed time 4 days, 10 hours, 27 seconds…
Global Data Strategy, Ltd. 2017
Modeling for BI Reporting – the Dimensional Model• A common way to model the data warehouse is Dimensional Modeling using a Star Schema, based
on methodology spearheaded by Ralf Kimball.
• For Dimensional Modeling, think of what you’re reporting “by” (e.g. by Month, by Region, etc.)• Dimensional modeling focuses on capturing and aggregating the metrics from daily operations that
enable the business to evaluate how well it is doing.
“What do I want to report by?” (Apologies to grammarians!), e.g.
by monthby regionby quarterby product
The lines on a dimensional data model represent navigation paths, not business rules.
Global Data Strategy, Ltd. 2017
The Star Schema
Dimension
Dimension
Dimension
Dimension
Dimension
Fact
Facts: Contain the actual values to be reported upon. e.g. Sales Figures • Few attributes (with links/keys to the dimensions)• Many values
Dimensions: Contain the details that describe the central fact. e.g. Month, Region, Quarter• Many attributes• Few values
Global Data Strategy, Ltd. 2017
The Star Schema• The following is a sample Star
Schema in a data modeling tool showing:
• Internet Sales (Fact) by:
• Time Period (Dimension)
• Promotion (Dimension)
• Product (Dimension)
• Customer (Dimension)
Global Data Strategy, Ltd. 2017
Summary
• The rise of the “Data Driven Business” has increased demand for BI reporting, particularly Self-Service Reporting
• BI Reporting is only as good as the underlying metadata, data structures, and data quality
• Data Models are a critical tool for• Understanding the business meaning of data
• Making BI Reporting more intuitive
• Improving the performance of BI queries
• Understanding source & target systems and the resultant data lineage
• Find a balance – “Model What Matters”• Modeling & metadata helps define key trusted data sets
• But not all data needs to be modeled – allow for exploration & discovery
Global Data Strategy, Ltd. 2017
Contact Info
• Email: [email protected]
• Twitter: @donnaburbank
@GlobalDataStrat
• Website: www.globaldatastrategy.com
• Company Linkedin: https://www.linkedin.com/company/global-data-strategy-ltd
• Personal Linkedin: https://www.linkedin.com/in/donnaburbank
30
Global Data Strategy, Ltd. 2017
About Global Data Strategy, Ltd
• Global Data Strategy is an international information management consulting company that specializes in the alignment of business drivers with data-centric technology.
• Our passion is data, and helping organizations enrich their business opportunities through data and information.
• Our core values center around providing solutions that are:• Business-Driven: We put the needs of your business first, before we look at any technology solution.• Clear & Relevant: We provide clear explanations using real-world examples.• Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s
size, corporate culture, and geography. • High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of
technical expertise in the industry.
31
Data-Driven Business Transformation
Business StrategyAligned With
Data Strategy
Visit www.globaldatastrategy.com for more information
Global Data Strategy, Ltd. 2017
DATAVERSITY Training Center
• Learn the basics of Metadata Management and practical tips on how to apply metadata management in the real world. This online course hosted by DATAVERSITY provides a series of six courses including:• What is Metadata
• The Business Value of Metadata
• Sources of Metadata
• Metamodels and Metadata Standards
• Metadata Architecture, Integration, and Storage
• Metadata Strategy and Implementation
• Purchase all six courses for $399 or individually at $79 each.Register here
• Other courses available on Data Governance & Data Quality
32
Online Training Courses
Metadata Management Course
Visit: http://training.dataversity.net/lms/
Global Data Strategy, Ltd. 2017
Lessons in Data Modeling Series
• January 26th How Data Modeling Fits Into an Overall Enterprise Architecture
• February 23rd Data Modeling and Business Intelligence
• March Conceptual Data Modeling – How to Get the Attention of Business Users
• April The Evolving Role of the Data Architect – What does it mean for your Career?
• May Data Modeling & Metadata Management
• June Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling
• July Data Modeling & Metadata for Graph Databases
• August Data Modeling & Data Integration
• September Data Modeling & MDM
• October Agile & Data Modeling – How Can They Work Together?
• December Data Modeling, Data Quality & Data Governance
33
This Year’s Line Up
Global Data Strategy, Ltd. 2017
Questions?
34
Thoughts? Ideas?