Date post: | 20-Aug-2015 |
Category: |
Technology |
Upload: | dataversity |
View: | 3,516 times |
Download: | 1 times |
Data Governance
IASA is
a non-profit professional association
run by architects
for all IT architects
centrally governed and locally run
technology and vendor agnostic
The
use,
disclosur
e, reproduction
, modification
, transfer, or
transmittal of thi
s work
without
the written permissio
n of IA
SA is
strictly prohibited
. © IA
SA 2009
Information Architecture
Module 0: Course Intro, Architecture Fundamentals Introduction: Data, Information and Knowledge
Module 1: Information for Business Lesson 1 – Information as Strategy Lesson 2 – Relating Information to Value Lesson 3 – Information Scope and
Governance
Module 2: Information Usage Lesson 1 – Who Uses Your Information Lesson 2 – How, When, Where and Why
is Information Used Lesson 3 – Form Factors Lesson 4 – Usage Design 1 Lesson 5 – Quality Attributes for
Information Architecture Lesson 6 – Data Tools and Frameworks
Module 3: Data Integration Lesson 1: Integrating at the Company Level Lesson 2: Data Characteristics Lesson 3: Data Integration at the System level
Module 4: Data Quality and Governance Lesson 1 – Data and Information Quality Lesson 2: Data Compliance Lesson 3 – Data and Information Governance
Module 5: Advanced Information Management Lesson 1: Data Warehousing Lesson 2: Business Intelligence Lesson 3: Data Security and Privacy Lesson 4: Metadata and Taxonomy Management Lesson 5: Knowledge Management
Module 6: Architecture throughout the Lifecycle
Typical State of Affairs
The Problem
What is data governance?
Governance is Not a Popular Topic
Governance is Overcomplicated
It is not about politics
It is not about delivering verdicts
It is not about defining solutions
Step 1: Form a Team
Step 2: Establish Lineage
Step 3: Assess Maturity
Step 4: Create Governance Body
Step 5: Define Governance
Step 6: Define Charter
•Charter
Step 7: Define Scope
Step 8: Create a Plan
Career Path
Iasa Architect Services
Adopt Iasa Standards – Skills taxonomy, role descriptions, compensation models
Set your goals for value
Assess your current team – gap analysis
Implement processes
Develop and educate people
Certify employees and vendors
Build communities
•December 6th – 7th, 2012 // Austin, TX
•Training: Dec. 3rd – 5th, 2012•Certification: Dec. 7th – 9th, 2012
•
•www.iasaworldsummit.org•Leading Innovation in Architecture. Stay ahead of the technology curve and shape the future of architecture in your organization.
•Connect and share insights with the largest global network of technology and enterprise architect practitioners.
•Attend sessions on the most current breakthroughs, case studies andkey topics in architecture - presented by a mix of practicing architects
from top performing businesses and organizations.
•Participate in pre-conference workshops and training. Maximize your time
at the summit by taking part in one of six intensive training courses designed
to provide immediate solutions to benefit your everyday practice.
•Featuring Industry Leading Keynote Speakers:
•Sheila JeffreySenior Information Architect
Bank of America
•David Del GiudicePrincipal Architect
AstraZeneca
•Paul PreissPresident, Founder
Iasa Global
•Cat SuschEnterprise Architect
Microsoft
•Scott WhitmireEnterprise Architect
Nordstrom
•David ManningIT Enterprise ArchitectIdaho Power Company
End Module
Data Architecture for Data Governance
Presented by Malcolm Chisholm [email protected]
Telephone 732-687-9283www.refdataportal.com
www.bizrulesengine.com
© AskGet.com Inc., 2012. All rights reserved
November 8, 2012
• Introducing Enterprise Architecture (EA)
• Introducing Data Governance
• Data Architecture and Data Governance
• Data Architecture Patterns
• Data Architecture, Governance, and The Business
Agenda
© AskGet.com Inc., 2012. All rights reserved
Introducing Enterprise Architecture (EA)
© AskGet.com Inc., 2012. All rights reserved
Need for Enterprise Architecture (EA)
Myriad of Independent Projects in IT
Here is where we want to be in 5
years…
CxO’s
Strategy is Set at High Level
Operationalization
Enterprise Architecture
…Others…
• EA is responsible for ensuring overall business strategy is implemented by IT
© AskGet.com Inc., 2012. All rights reserved
The IT Mindset
© AskGet.com Inc., 2012. All rights reserved
IT CxO
That’s not a clear requirement…
Tell me what you want me to buildThen I will design itThen I will build itThen I will turn it over to youThen I will walk away
We have a high-level strategy…
• IT people just want to build stuff and hand it off to other people•They want business sponsors who will pay for this stuff and tell them exactly what is needed
•Strategy does not match this mindset
Design without Architecture
© AskGet.com Inc., 2012. All rights reserved
• The Winchester Mystery House• Lots of design – but no architecture• There is a difference between architecture and design
Rooms: 160; Doorways: 467; Doors: 950; Fireplaces: 47 (gas, wood, coal); Bedrooms: 40
Constructed 1884 – 1922 (38 continuous years); Cost: $5.5M
Blueprints: Never made; Individual rooms sketched out by Sarah Winchester on paper or other media (e.g. tablecloths)
All Design – No Architecture
“Circular” Stairway
Stairway to Ceiling
“…staggering amount of creativity, energy, and expense poured into each and every detail”
http://www.winchestermysteryhouse.com
Design vs. Architecture
• Architecture deals with many instances of a component type that must interact
• Design deals with one instance of a component type, without regard to interaction
• E.g. Perspective of Databases : Data Environment (BI or Integration Environment in this example)
DDL
Source Mirror
IntegrationChanged
Data Capture
Data Flow
© AskGet.com Inc., 2012. All rights reserved
Basic EA Taxonomy (“BAIT”)
Enterprise Architecture
© AskGet.com Inc., 2012. All rights reserved
Business Architecture
Application Architecture
Information Architecture
Technology Architecture
• EA is clearly not a monolithic discipline• It looks more like a collection of dissimilar components • Each component is a competency – so this matters• Yet there is no common agreement
Introducing Data Governance
© AskGet.com Inc., 2012. All rights reserved
What is Data Governance? Starter Definitions
© AskGet.com Inc., 2010. All rights reserved
”The exercise of decision-making and authority for data-related matters.”
”A system of decision rights and accountabilities for information-related processes, executed according to agreed-upon models which describe who can take what actions with what information, and when, under what circumstances, using what methods.”
www.datagovernance.com
From the Data Governance Institute
Data Governance is about Processes
© AskGet.com Inc., 2010. All rights reserved
Deciding who has what rights regarding data
In other Words…
Processes for…
Making decisions about data
Implementing decisions about data
Identifying the processes needed to manage data
Identifying the actors in these processes
Designing these processes
And designing the processes to design these processes
W R ON G
Data Governance – A Caution
© AskGet.com Inc., 2010. All rights reserved
The Business has…. IT has…
Business Processes Governance
Data Governance is the design, building, and operation of formal business processes to manage the enterprise data resource.
• The decision rights of actors are defined within the context of these business processes • There is no real distinction between the processes of “data governance” and the “business processes” of the business . [Compare HR].• A distinction that identifies the set of business processes that manage the enterprise data resource as “data governance” is valid. [Compare HR].• Use of the term data governance to further the pretence that IT is universe totally outside of the enterprise is invalid.• Therefore, the processes of data governance are truly business processes
Data Architecture and Data Governance
© AskGet.com Inc., 2012. All rights reserved
Architectural Levels
Reference Architecture
Conceptual Architecture
Logical Architecture
Physical Architecture
Assimilate best architectural
practices from the IT industry. All
relevant component classes and patterns are understood.
Match/select architectural
component classes and patterns to the
business needs, (including future
plans). Influencing actual physical
implementations to ensure
conformance with logical architecture
Detailed design of future state
architecture.
A reference architecture is the highest level of architecture. It can be undertaken prior to even thinking about the enterprise. Its advantages are that it can be very high level and can cover the entire enterprise. Useful for thinking about high level data layers
© AskGet.com Inc., 2011. All rights reserved
Reference Data Architecture - Example
© AskGet.com Inc., 2011. All rights reserved
• Example of a simple reference data architecture• Just data layers – no services or componentsBUT – just having a picture can be problematic
Need Definitions, Explanations – Not Just Picture
LayerCharacteristic Transactional
Application Data Warehouse Data Mart
Data produced via the automation of business processes
View of data across the enterprise. Supports dissemination, derivation of knowledge and history
Purpose
Data Life Cycle
Data Operations
Data Model
Data structured and filtered to support specific information needs of small groups of users.
All base (non-derived) data originates here
Derivations (including aggregations) produced here, and history is inferred
Data from Warehouse is transformed to support specific reporting
Create / Source / Read / Update / Delete / Archive
Extract / Transform / Load / Derive / Publish / Archive
Subscribe / Transform / Archive
Normalized to 3NFSubject Oriented / Snowflaked / Conformed Dimensions
Information Requirement Oriented / Snowflaked / Conformed Dimensions
© AskGet.com Inc., 2011. All rights reserved
•Much more is needed than the above•Definitions are a technical reference; explanations help stakeholders to understand the reference architecture
Subject Area Model
© Askget.com Inc., 2011. All rights reserved
• A Subject Area Model is a taxonomy of the major areas in the enterprise that are relevant to data management• A necessary starting point for MDM• 1 – A subject area should have stable definitions of entities• 2 – Production of master data should not cross subject area boundaries• Identify the major data concepts in each subject area – may will be master data entities.
A Subject Area Model is Not Costly
© Askget.com Inc., 2011. All rights reserved
• 10-15 subject areas per model is the norm• Do not need elaborate data modeling tools• Much more about a standardized view of the enterprise• Can be done with 1-3 people• Gets a lot of bang for the buck• Is high-level and important
Data Management
Example of Use: Establish Vendor Data Profiles
© AskGet.com Inc., 2011. All rights reserved
• A Subject Area Model provides the basis for a common understanding of the data vendors provide
• Will need to go to major entity clusters within Subject Area
Data Management
Data Vendor 1
Data Vendor 2
Data Vendor 3
Data Vendor 4
Data Vendor 5
Subject Area Model(s)
Create & Manage Subject Area Model(s)
Users
Map Vendor’s Data to Subject
Areas
Map Subject Areas to User Requirements
Map Vendors to Uses
A Subject Area Model is a Taxonomy
CLIENT MARKETINGAND MANAGEMENT
CLIENT
CONTACT
MEETING
SERVICE REQUEST
PLANCONTRIBUTION /
WITHDRAWAL
ADDRESS
© Askget.com Inc., 2011. All rights reserved
• People think a taxonomy is a way of unlocking the secrets of the universe• Taxonomies are a tools – you can have as many as you need to do your job • You can never expect the boundaries of a subject area model to be exactly precise and never changing
Your model does not tell me what is
financial data!
I don’t like your definition of this subject
area!
You mean to tell me every one of these
subject areas includes access security for data!
Create a Metadata Subject Area Model
© Askget.com Inc., 2011. All rights reserved
The Fundamental Ontology of Metadata for the Enterprise
Metadata Subject Area Model
© Askget.com Inc., 2011. All rights reserved
The Subject Area Model is itself Metadata and needs to be managed
Advantages
1. Standardizes the metadata in the enterprise at a high level.
2. Can be used to prioritize projects and programmes.
3. Useful in communications.
4. Shows what metadata is being produced by subject area. This is a high-level inventory.
Disadvantages
1. It is a taxonomy and can be argued over. No one taxonomy will satisfy every perspective. We use data categorization for that (see later). The Subject Area Model should be the most common and natural perspective.
2. The Subject Area Model often has more things expected of it than it can deliver. Managing expectations can be tricky.
5. Can distinguish data-related metadata from other data
6. Within each subject area, definitions should be constant. There may be variations across subject areas.
7. Subject areas are candidates for conceptual models.
Data Architecture Patterns
© AskGet.com Inc., 2012. All rights reserved
What is an Architectural Pattern?
© AskGet.com Inc., 2011. All rights reserved
• Patterns occur a lot in architecture• There are also styles, metaphors and antipatterns• Seem to have emerged from the application architecture
competency (e.g. rare to hear of dimensional modeling as a pattern or style)
“Pattern in architecture is the idea of capturing architectural design ideas as archetypal and reusable descriptions”- Christopher Alexander, quoted in Business Model Generation by Osterwalder and Pigneur (2010)
• An architectural style is a central, organizing concept for a system.
• An architectural pattern describes a coarse-grained solution at the level of subsystems or modules and their relationships.
• A system metaphor is more conceptual and it relates more to a real-world concept over a software engineering concept.
- Quoted from A Practical Guide to Enterprise Architecture by McGovern, Ambler et al, at
http://www.infoq.com/news/2009/02/Architectural-Styles-Patterns
Antipattern: A pattern that is commonly used, but which is ineffective or damaging. E.g. Analysis Paralysis. - Term created by Andrew Koenig, 1995
Example: Farm and Market Pattern
© AskGet.com Inc., 2011. All rights reserved
MDM HUB(“MARKET”)
Publish
App 1
App 2
App 3
App ...
TRANSACTION APPLICATIONS
App A App B App ...
OTHER CONSUMING APPLICATIONS
Other Functionality
Master Data Entity 1 App
Master Data Entity 2 App
Master Data Entity 3 App
Master Data Entity N App
MASTER DATA PRODUCTION (“FARMS”)
Reduced Cleansing
Reduced Integration
MDM HUB
Cleanse
Integrate
Publish
App 1
App 2
App 3
App ...
TRANSACTION APPLICATIONS
App A App B App ...
OTHER CONSUMING APPLICATIONS
Produce Master Data
Other Functionality
Steward
Traditional MDM Integration Hub
Farm and Market Pattern for MDM
Six Patterns for Data Hubs: 1 – Publish/Subscribe
© AskGet.com Inc., 2011. All rights reserved
• Publisher pushes data to hub• Subscriber pulls data from hub• No data integration• Publisher may not know who the subscribers are• Weak governance - may not even be SLA’s
Six Patterns for Data Hubs: 2 – ODS for Integrated Reporting
© AskGet.com Inc., 2011. All rights reserved
• Supports principle that transaction applications will not do operational reporting (adverse affect on performance)
• Some form of integration• Anti-pattern of real-time data warehousing• History (changed data capture) not considered
Six Patterns for Data Hubs: 3 – ODS for Data Warehouses
© AskGet.com Inc., 2011. All rights reserved
• No reporting in this kind of ODS• Probably evolved from ODS for reporting• Strong integration• Just to feed warehouses • At odds with Data Warehouse that has own staging, integration
Six Patterns for Data Hubs: 4 – Traditional MDM Hub
© AskGet.com Inc., 2011. All rights reserved
• Master Data only• Integration does happen (typically just Trust & Survivorship)• Some content management needed – typically to correct data
quality problems• Typically DQ functionality an afterthought• Distributes “Golden Copy” Master Data to enterprise
Six Patterns for Data Hubs: 5 – Message Hub
© AskGet.com Inc., 2011. All rights reserved
• Messages – real-time or near-real-time - not batch data• Message switch • Command and control for message orchestration• Not just “listening” – does routing• Messages also stored in database
Six Patterns for Data Hubs: 6 – Integration Hub
© AskGet.com Inc., 2011. All rights reserved
• Like message hub, but for batch data• Event data equivalent of MDM hub• Just one place where integration done• Both warehouse and transaction applications need integrated data• Removes need for each application to do integration by itself
Data Architecture, Governance, and The Business
© AskGet.com Inc., 2012. All rights reserved
Another View of EA
• Long ago, people had to know the business rules concerning what they did
• These are now in applications, and staff are more concerned with knowing how to make the applications work
• Data is an asset, but staff tend to know little about the data• Loss of business knowledge
Many Years Ago……”the business” knew the business
Today……it does so much less
© AskGet.com Inc., 2012. All rights reserved
Who Knows The Business?
• The business seems to be more fragmented than long ago (though that is difficult to prove)
• Managers who know the business know less of it and are so time-constrained they cannot help much on aligning projects to enterprise strategy
© AskGet.com Inc., 2012. All rights reserved
Populate EA with Business Experts
• Business experts provide better relations with rest of business• Keep architects with just being oriented to architecture • Immediate business credibility for EA• They get poached
© AskGet.com Inc., 2012. All rights reserved
A Different EA Vision:Architects + Business Experts
IT Tendency:Populate EA with Architects
Questions and Answers
© AskGet.com Inc., 2012. All rights reserved
Presented by Malcolm Chisholm Ph.D.Telephone 732-687-9283
Data Architecture for Data Governance
November 8, 2012