Top 5 Data Architecture Challenges with Ron Huizenga

EMBARCADERO TECHNOLOGIESEMBARCADERO TECHNOLOGIES

Top 5 Data Architecture

Challenges

Ron HuizengaProduct Manager – ER/[email protected]

EMBARCADERO TECHNOLOGIES

5 Challenges

• Evolution of methodologies & culture

• Adapting to changing architecture

• Complex data environments

• Data quality

• Business focus

2


Evolution of Methodologies & Culture

3

• Traditional/Waterfall (up to 1999)– Big design up front– Rigid organizational structure

• Agile – The Rebellion? (1996 – present)– Incremental, time boxed delivery– Self organizing team (who needs roles?)

• The Hybrid (200x? – present)– Agile operational approach– Leverage individual roles/skill sets– Importance of architecture


Agile Process

4


ER/Studio – Agility Support• True multi-level sub-models• Repository

– Check-out/Check-in of sub-models– Individual data objects– Named versions– Annotation

• Reverse Engineering• Compare and Merge• Naming standards capabilities• Macros• Collaboration & publishing• Data interchange bridges

5


Adapting to Changing Architecture

• Relational database

• Object oriented

• Unstructured data

• Document management

• Service oriented architecture

• Big data

6


Service Oriented Architecture (SOA)

Publish & Subscribe

Federated Queries

Operational Data Store

Data Warehouse

Unstructured Data & DocumentsMetadata

Repository

Authentication& Security Services

Application ServerBusiness Process

ManagementBusiness Rule Management

Complex Event Processing Lifecycle Management Policy EnforcementBusiness Activity

MonitoringAdapters Communications Messaging

Metadata ManagementMaster Data Management

Transformation Canonical Model Canonical Messaging Data Integration

Data Federation Data Access Services Document ServicesContent Management

Services

Enterprise Service Bus

Mapping & Transformation

Business Objects

7


Big Data

• Volume

• Velocity

• Variety

• Veracity

8


ER/Studio – Native Big Data Support

• MongoDB– Versions 2.4 and 2.6

• Hadoop Hive– Versions 0.12 and 0.13

• Capabilities– Diagramming

– Reverse Engineering (JSON, BSON)

– DDL supported for Hive

9


ER/Studio – Big Data Notation Enhancement

• Physical Model

– Objects instead of Tables

• Nested Objects

– “Is Contained In” relationship type

10


Complex Data EnvironmentsEvolution:

• 38 years of construction

• 147 builders

• No Blueprints

• No Planning

Result:

• 7 stories

• 65 doors to blank walls

• 13 staircases abandoned

• 24 skylights in floors

• 160 rooms, 950 doors

• 47 fireplaces, 17 chimneys

• Miles of hallways

• Secret passages in walls

• 10,000 window panes (all bathrooms are fitted with windows)

11


Complex Data Landscape

12

• Proliferation of disparate systems

• ERP, mismatched departmental solutions

• SAAS (externally controlled and managed)

• Obsolete legacy systems

• Poor decommissioning strategy

• Point-to-point interfaces

• Data warehouse, data marts, ETL …


ER/Studio – Conquering Landscape Complexity• True multi-level sub-models (hierarchy)

• Reverse engineering (extensive list of platforms)

• Comprehensive metadata extensions (attachments)

• Naming standards capabilities

• Universal mappings

• Macros

• Data lineage

• Business Architect – data context (processes)

• Repository, collaboration & publishing

13


ER/Studio – Compare and Merge example

14


C&M cont.

15


16

C&M cont.


17

C&M cont.


18

C&M cont.


19

C&M cont.


20

C&M cont.


Data Quality

21

• Accuracy

• Timeliness

• Completeness

• Consistency

• Relevance

• Fitness For Use


Poor Data Quality Implications

22

• Costs a typical company the equivalent of 15% to 20% of revenue– Estimated by US Insurance Data Management Association

• Low Quality = Low Efficiency

• It is insidious – most data quality issues are hidden in day to day work

• From time to time, a small amount of bad data leads to a disaster of epic proportions


23

Space Shuttle

Challenger

QueCreek

Mine Flooding

When Data Flaws Happen…


Poor data quality isn’t a new problem

24


Mitigation Best Practice

25

• Adopt the philosophy of prevention

• Show thought leadership

• Be accountable at the points of data creation

• Measure, control, improve

• Establish data culture


Technique: Attachments (Metadata extensions)

26


Business Focus

• Governance

• Champion for data value, data quality

• Consultative approach – business acumen

• Establish a data culture

• Business collaboration

27


Addressing Governance through Models

28

Data Governance

Data Architecture Management

Data Development

Database Operations

Management

Data Security Management

Reference & Master Data Management

Data Warehousing

& Business Intelligence

Management

Document & Content

Management

Metadata Management

Data Quality Management


Data Management

29

• Shared responsibility and collaboration

– Data Management Professionals

– Business Data Stewards

• Cultural change leadership

– Adopting the principles and practices of data management requires leadership from change agents at all levels


• Powerful enterprise glossary, model & metadata collaboration

• Integrate key business terms and definitions with business systems

• View, store, and manage a single source of business definitions

• Attach business policies to daily workflows with contextual alerts and tips

Business Collaboration

30


Concluding Remarks

31


Thank you for attending!

• Learn more about the ER/Studio product family: http://www.embarcadero.com/data-modeling

• Trial Downloads: http://www.embarcadero.com/downloads

• To arrange a demo, please contact Embarcadero Sales: [email protected] 1 (888) 233-2224

32

http://www.embarcadero.com/data-modeling

http://www.embarcadero.com/downloads

mailto:[email protected]

Date post:	25-Dec-2014
Category:	Software
Upload:	embarcadero-technologies
View:	101 times
Download:	0 times

Top 5 Data Architecture Challenges with Ron Huizenga

Software