A Global Open StandardFor Building
Your Controlled Vocabulary
Ron Schuldt, ChairRon Schuldt, Chair
The Open Group UDEF ProjectThe Open Group UDEF ProjectJuly 13, 2009July 13, 2009
UU DD EE FF
AgendaAgenda
• The Problem
• The Missing Metadata Layer
• A Global Metadata Managed Architecture
• ISO/IEC 11179 – Metadata Registry Standard
• Universal Data Element Framework (UDEF) Overview
• Leveraging Online Gap Analysis
• Sample Controlled Vocabularies
• The Need for Distributed UDEF
• Example Distributed UDEF Use Cases
• UDEF Features Summary
• Some UDEF Tools
Enterprises Want ThisEnterprises Want This
External “In” Space
Internal Space
External “Out” Space
Procuring
ManufacturingLegalFinanceAssembling
Customer Support
ProcurementSystems
DesignSystems
Online Systems
ERPSystems
RequirementsSystems
Systems
Processes
But Experience This*
* Extract from The Open Group SOA White Paper
At present, this problem is typically addressed within the services, and by use as far as possible of enterprise-standard and industry-standard vocabularies. In the future, it may be addressed by semantic technology that is incorporated in the infrastructure, which will be a more scalable way to deliver the ideal of Boundaryless Information Flow.
Problem – Global PerspectiveProblem – Global PerspectiveEach organization is attempting to set its own semantics standardEach organization is attempting to set its own semantics standard
Each must interface with organizations they do not controlEach must interface with organizations they do not control
The problem is the The problem is the lack of a controlled vocabularylack of a controlled vocabulary
Customers
Suppliers
Partners
Other Retail
Local
State
TransUtilities
Banks
Federal
Organization
Problem – Internal PerspectiveProblem – Internal Perspective
<PARTNUMBER>111-222-333</PARTNUMBER>
<partNumber>111-222-333</partNumber>
<PartNumber>111-222-333</PartNumber>
<partnumber>111-222-333</partnumber>
Though semantically equal, the following are 4 different XML tag names
App B App C
App A
Other Apps
Legacy Data
Conflicting semantic overlaps between back-office systems
The Goal
Current Point-to-Point Approach --- n(n-1)
Global Semantics Standard Approach --- 2n
$$
Savings
GlobalSemanticsStandard
0
50
100
150
200
250
300
350
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Reduce Requirements and Design-Time Phase Semantics Analysis Time and Cost
Data-Information-Knowledge-WisdomData-Information-Knowledge-Wisdom
Data - Information Reference Model
• Databases • Data Formats
• Information discovery • Information retrieval
• File Servers • Repositories
• Naming Conventions • Indexing Methods• Registry
• Ontologies• Taxonomies • Vocabularies
• Search engines • Inference engines• File retrieval
• Data standards • Middleware• DBMS
• Vaults • Format standards• Change control
Structured DataStores
Un-Structured DataFiles
Data - Metadata Management
Data Management
Information - Metadata Management
Information Management
• Logical Data Model • File Labels
Analogous to the OSI 7 layer reference model
Example functions and technologies for each layer
Data - Information Reference Model
• Databases • Data Formats
• Information discovery • Information retrieval
• File Servers • Repositories
• Naming Conventions • Indexing Methods• Registry
• Ontologies• Taxonomies • Vocabularies
• Search engines • Inference engines• File retrieval
• Data standards • Middleware• DBMS
• Vaults • Format standards• Change control
Structured DataStores
Un-Structured DataFiles
Data - Metadata Management
Data Management
Information - Metadata Management
Information Management
• Logical Data Model • File Labels
Analogous to the OSI 7 layer reference model
Example functions and technologies for each layer
The layer typically overlooked
Wiki - Controlled Vocabularies
“Controlled vocabularies provide a way to organize knowledge for subsequent retrieval. They are used in subject indexing schemes, subject headings, thesauri and taxonomies. Controlled vocabulary schemes mandate the use of predefined, authorized terms that have been preselected by the designer of the vocabulary, in contrast to natural language vocabularies, where there is no restriction on the vocabulary.”
The above provides a good description of the function that the UDEF offers to any enterprise on the planet since the UDEF is open and is infinitely extensible and can transcend any domain or function of any enterprise.
The Open Group Architecture Framework (TOGAF)The Open Group Architecture Framework (TOGAF)
The Open Group Vision Statement
The Open Group is a vendor-neutral and technology-neutral consortium,
whose vision of Boundaryless Information Flow™ will enable access to integrated information, within and among enterprises, based on open standards and global interoperability.
TOGAF does a great job with Enterprise Architecture but does not address Boundaryless Information Flow
TOGAF v8.1 Principle 10
Statement
“Users have access to the data necessary to perform their duties; therefore, data is shared across enterprise functions and organizations.”
Rationale
“… The speed of data collection, creation, transfer, and assimilation is driven by the ability of the organization to efficiently share these islands of data across the organization. …”
Implications
“… need to develop standard data models, data elements, and other metadata that defines this shared environment and develop a repository system for storing this metadata to make it accessible. …”
Chapter 29, Data is Shared
Metadata Management
Metadata describes the characteristics of an Metadata describes the characteristics of an artifact, such as its name, type of content, artifact, such as its name, type of content, sensitivity, lifecycle phase, perceived sensitivity, lifecycle phase, perceived importance to specific audiences, quality or importance to specific audiences, quality or value to the enterprise, and its relationships to value to the enterprise, and its relationships to other artifacts. other artifacts.
Managed Metadata with a Managed Metadata with a Controlled Vocabulary Controlled Vocabulary … Standard Metadata… Standard Metadata Will enable organization to reduce the costs Will enable organization to reduce the costs
of building and maintaining application of building and maintaining application interfacesinterfaces
Is a necessary enabler of a Service Oriented Is a necessary enabler of a Service Oriented Architecture (SOA)Architecture (SOA)
Will improve data and information assets Will improve data and information assets discovery and reuse … data miningdiscovery and reuse … data mining
Will improve knowledge worker productivityWill improve knowledge worker productivity
To achieve enterprise IT agility requires effective metadata management
Invoice No.
Qty
Total Price
Order Date
Buyer
1234
10
9/20/07
Metadata Data
John Doe
$210.00
Metadata Registry ISO/IEC 11179 Standards
Part 1: Metadata Registries - Framework
Part 2: Metadata Registries - Classification
Part 3: Metadata Registries - Registry Metamodel and Basic Attributes
Part 4: Metadata Registries - Formulation of Data Definitions
Part 5: Metadata Registries - Naming and Identification Principles
Part 6: Metadata Registries - Registration
http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html/
ISO/IEC 11179-5 Summary*
• Provides guidance for the naming or identification of data constructs administered in a metadata registry.
• Names are applied to data constructs through the use of naming conventions. There are semantic, syntactic and lexical rules used to form a naming convention.
• Identifiers are designations meant for dereferencing data constructs for metadata management and exchange.
*Extracts from ISO/IEC 11179-1:2004(E) – section 8.1.5ISO/IEC 11179-1:2004(E) – section 8.1.5
ISO/IEC 11179 – Terminology
DataElementConcept
DataElement
ValueDomain
ObjectObjectClassClass
PropertyProperty RepresentationRepresentation
CoreData
Element
ApplicationData
Element
The “Key” for a Controlled Vocabulary of Structured Data
A Metadata Managed Architecture
EAI
Transformation Engines
Interfaces to Back-Office
Systems
• Data Dictionary
• Mapping Matrices
• Std XML Schema
UDEF-Indexed Metadata Registry/Repository
InterfaceDevelopers
Run Time
Data ModelersAnd Apps Developers
Design Time
Internet
UDEFExtension Process
Global UDEF Registry
Vendors with Canonical Models
Software Vendors
with UDEF IDAPIs Web
Public
Extend Matrices
UseMatrices
Std Schema
UDEF-Indexed Metadata Registries
Build/Extend Schema
Centralized metadata registry/repository
• Enables reuse to reduce costs
• Encourages standardization
UDEF Thumbnail Sketch
The Open Group’s Universal Data Element Framework (UDEF)
The only truly open standard that will simplify building an enterprise-wide controlled vocabulary for structured data and facilitate normalization across data models while enabling SOA I/O interoperability across all services that adopt UDEF through global indexing keys
Oracle’s Daniel Serain recommends using UDEF global
indexing keys for SOA services I/O
UDEF is an Open Standard that Enables Interoperability
Universal Data Element FrameworkUniversal Data Element Framework
UDEF names follow the rules of English – qualifiers precede the word they modify
Data Element Concept Name
Object Class Term
0...n qualifiers +1 or more required
Object Class
+
Example UDEF-Based Data Element Concept Names
Document Abstract Text
Enterprise Name
Product Price Amount
Product Scheduled Delivery Date
Engineering Design Process Cost Amount
Employee Person Family Name
UDEF ObjectClass List• Entity• Document• Enterprise• Place• Program• Product• Process• Person• Asset• Law-Rule• Environment• Condition• Liability• Animal• Plant• Substance• Event
Property Term
0..n qualifiers +1 required Property
Property List*• Amount• Code• Date• Date Time• Graphic• Identifier• Indicator• Measure• Name• Percent• Picture• Quantity• Rate• Text• Time• Value• Sound• Video
ISO/IEC 11179-5 Naming Convention
UDEF is a proposed universal instantiation of ISO/IEC 11179-5
* Based on Tables 8-1 and 8-3 in ISO 15000-5
Creating UDEF IDs
UDEF Trees
17 Object Class Trees 18 Property Trees
Entity Asset Person Amount Name… …
Patientau
50 1
Family Alias…Country …
101…
1 11 40
Patient Person_Family Name has UDEF ID = au.5_11.10
See http://www.opengroup.org/udefinfo/defs.htm
Employee …
a
…
UDEF Provides The Indexing Key
External
Conceptual
Physical
System A
External
Conceptual
Physical
System B
Registry Derived Name
Emp_Start Empl-Hire-Dt
EmpStr EmplHr
a.5_49.6
Registry Derived IDRegistry Derived ID
Open Group UDEF Registry
Map to registry
Assign UDEF ID alias
UDEF Object Term identifies the table
UDEF Property Term identifies the column
Employee PERSON
Hired DATE
UDEF Object Term
UDEF Property Term
UDEF – Current Status
All enterprisesAll enterprises
use the UDEFuse the UDEF
Controlledextensionprocess
Baseline UDEFin usable form
Support by keyvendors (IBM,
SAP, Oracle . .)
Early adoptersuse the UDEF
Training
Demonstratedbenefits
Promotion
Pilot – keyvendor(s) &end user(s)
Global Registry
HomelandSecurity
& Electronic HealthRecord
Launch
Ready
Started
UDEF in other languages
Disaster Response
Initial
UDEF ID Object Type or Role Object Class Property Type Property
a.5_51.6 Birth
HR ExampleEmployee
PersonEmployee
Number a.5_12.35.8
Date
Hire Date
Last Name
First Name
Birth Date
a.5_49.6
a.5_51.6
a.5_12.10
a.5_11.10
6. Derive a structured ID based on the UDEF taxonomy that carries the UDEF inherited indexing scheme. For example <CustomerPersonFamilyName UDEFID=“as.5_11.10”>
Create UDEF Mappings
d.t.2_8
OAG Purchase Order: xCBL Purchase Order:
d.t.2_8
Map Both Systems and/or Both Standards Map Both Systems and/or Both Standards Independently to the UDEFIndependently to the UDEF
Source and Target Files
OAGIS StandardOAGIS Standard xCBL StandardxCBL Standard
Leveraging Online Tool
Vocabulary GapSoftware
On-the-Wire Java and XSLT
SourceSystem
TargetSystem
Source XML Instance Doc with UDEF IDs
Target XML Instance Doc with UDEF IDs
SOAP SOAPDHTML
On-line Gap Analysis Report
Online Gap Analysis Tool – Online Gap Analysis Tool – https://jserver.opengroup.org/UDEF/UdefReport1
Gap Analysis Results
OAGIS StandardOAGIS Standard xCBL StandardxCBL Standard
Gap Analysis Results - EnhancedOAGIS StandardOAGIS Standard xCBL StandardxCBL Standard
HR Controlled Vocabulary Examples
For employee identification, the UDEF based data element concepts’ names and IDs are:
UDEF Name = Controlled VocabularyUDEF Name = Controlled Vocabulary UDEF IDUDEF ID
Employee Person Employer Assigned Identifier a.5_12.35.8
Employee Person Hired Date a.5_49.6
Employee Person Family Name a.5_11.10
Employee Person Given Name a.5_12.10
Employee Person Gender Code a.5_3.4
Employee Person Birth Date a.5_51.6
All HR applications on the globe could use “a.5” as the indexing ID for identifying the application table that contains employee data. The portion of the UDEF ID that starts with the underscore (i.e., _12.35.8, _11.10, _3.4, etc.) provides the indexing ID to identify the associated column in the HR table.
Procurement Controlled Vocabulary
For employee identification, the UDEF based data element concepts’ names and IDs are:
UDEF Name = Controlled VocabularyUDEF Name = Controlled Vocabulary UDEF IDUDEF ID
Purchase Order Document IdentifierPurchase Order Document Identifier d.t.2_8d.t.2_8
Purchase Order Document DatePurchase Order Document Date d.t.2_6d.t.2_6
Purchase Order Document Line Item IdentifierPurchase Order Document Line Item Identifier d.t.2_17.8d.t.2_17.8
Ordered Product IdentifierOrdered Product Identifier z.9_8z.9_8
Ordered Product Ordered QuantityOrdered Product Ordered Quantity z.9_13.11z.9_13.11
Ordered Product Total Price AmountOrdered Product Total Price Amount z.9_6.2.1z.9_6.2.1
All Procurement applications on the globe could use “d.t.2” as the indexing ID for identifying the application table that contains purchase order data. The portion of the UDEF ID that starts with the underscore (i.e., _8, _17.8, _6, etc.) provides the indexing ID to identify the associated column in the purchase order table.
The Need for Distributed UDEFThe Need for Distributed UDEF
Distributed UDEFDistributed UDEF
UDEF-tree
Descendant tree
Descendant tree
connectors
Global UDEFIndustry UDEF
Private UDEF
Example Industry SDEF MappingsExample Industry SDEF Mappings
Example Private UDEF ScenarioExample Private UDEF Scenario
UCAV
MH-60
AWACSF/A-18E/F
SOF
F-22
CAOC
SGU
U S TSDisplays
RC-135
Global Hawk - SAM
Global Hawk - SCUD
Global Hawk - CSAR
PDEF Scenario – Mission AssignedPDEF Scenario – Mission Assigned
UCAV
F/A-18E/F
Global Hawk - SAM
CAOC
TST Cell
SGU
U S TSDisplays
RC-135
(2)TransmitSAM sitetarget info
SAM
(1) Picks up
SAM siteson RWR
(5)Provide image
(6)CAOC displays SAM site image
(7) Task F/A-18E/F to
destroy SAM sites (ATO)
(8)ATO Ack
(4)ATO Ack.
(9) Task UCAV to
destroy SAMsites (ATO)
(10)ATO Ack
(3)Task Global Hawk
to providetarget area
images (ATO)
PDEF Scenario – Accomplish MissionPDEF Scenario – Accomplish Mission
UCAVCAOC
TST Cell
SGU
U S TSDisplays
SAM
(15)Provide image/video
(16)CAOC displays
SAM siteimage/video
(11)UCAV drops
bombs onSAM site
(13)Task Global Hawk
to performbomb damageassessment
(14)ATO Ack.
Global Hawk - SAM
F/A-18E/FRC-135
(12) Bomb dropped on SAM site
(12) Bomb dropped on SAM site
Example PDEF MappingsExample PDEF MappingsSee the UDEF Asset tree and UDEF Measure tree, respectively.
1) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with
UDEF ID = a.a.UDEF-P.a.o.1_2.11.13
The UDEF ID “a.a.UDEF-P.a.o.1_2.11.13” would be transmitted with the applicable latitude value to the applicable command center receiver and would be interpreted as
Tail-ID-1234.RC-135.Military.Aircraft.Asset_Latitude.Coordinate.Measure with the associated latitude value included within the message. The private portion of the UDEF name “Tail-ID-1234.RC-135” and associated private ID “a.a.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Military.Aircraft.Asset.
Other examples include the following:
2) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with
UDEF ID = a.a.UDEF-P.a.o.1_3.11.13
3) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Altitude.Coordinate.Measure with
UDEF ID = a.a.UDEF-P.a.o.1_5.11.13
4) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with
UDEF ID = f.b.UDEF-P.a.o.1_2.11.13
Example PDEF MappingsExample PDEF Mappings5) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with
UDEF ID = f.b.UDEF-P.a.o.1_3.11.13
6) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_JDAM.Bomb.Available.Quantity with
UDEF ID = f.b.UDEF-P.a.o.1_1.1.UDEF-P.2.11
7) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Available-Fuel.Weight.Measure with
UDEF ID = f.b.UDEF-P.a.o.1_1.UDEF-P.8.13
The private portion of the UDEF name “JDAM.Bomb” and associated private ID “1.1.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Available.Quantity. Similarly, the private portion of the UDEF name “Available-Fuel” and associated private ID “1.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Weight.Measure.
8) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Latitude.Coordinate.Measure with UDEF ID = a.a.UDEF-P.i.1_2.11.13
9) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Longitude.Coordinate.Measure with UDEF ID = a.a.UDEF-P.i.1_3.11.13
The private portion of the UDEF name “Enemy.Surface-To-Air-Missile-Launcher” and associated private ID “a.a.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Vehicles.Asset.
10 )Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with
UDEF ID = a.x.UDEF-P.a.o.1_2.11.13
Example PDEF MappingsExample PDEF Mappings11) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with
UDEF ID = a.x.UDEF-P.a.o.1_3.11.13
12) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Altitude.Coordinate.Measure with
UDEF ID = a.x.UDEF-P.a.o.1_5.11.13
The private portion of the UDEF name “Enemy.MIG-29” and associated private ID “a.x.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Military.Aircraft.Asset
13) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Target-Engagement.UDEF-P.Type.Code with
UDEF ID = f.b.UDEF-P.a.o.1_2.UDEF-P.33.4
Example target-engagement code values might be:
1 = Air-To-Surface
2 = Air-to-Air
The private portion of the UDEF name “Target-Engagement” and associated private ID “2.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Type.Code.
14) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Destroyed.Photo.Picture with
UDEF ID = a.a.UDEF-P.i.1_1.UDEF-P.1.3
15) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_ Destroyed.Photo.Picture with
UDEF ID = a.x.UDEF-P.a.o.1 _1.UDEF-P.1.3
Some UDEF FeaturesSome UDEF Features
• Conforms to ISO/IEC 11179-5
• Establishes global indexing scheme for data element concepts analogous to Dewey Decimal System used in libraries to index books
• Enables normalization of data models on global scale
• Viewable for anyone with a browser
• Online gap analysis utility to anyone with a browser
• Multi-lingual (German, French, Chinese soon) in addition to English baseline and Dutch equivalent
• Unlimited extensibility
• Anyone with an email address can propose extensions
• Downloadable in XML and RDF formats to anyone with an email address
• Supports industry or organization unique extensions in a manner analogous to Internet DNS
Free UDEF ToolsFree UDEF Tools
• Free Tools
• Online Gap Analysis Tool on The Open Group Web Site
• Download sample source and target XML files for purchase order based on two different standards
See Gap Analysis section of http://www.opengroup.org/udefinfo/
• Obtain gap analysis reporthttps://jserver.opengroup.org/UDEF/UdefReport1
• UDEF Explorer Mapping Tool by Knotionhttp://www.knotion.net/index.php?option=com_content&task=view&id=68&Itemid=76
For Additional Information
OPENGROUP UDEF Web Site – to download the UDEF
http://www.opengroup.org/udefinfo/
The UDEF On-Linehttp://www.opengroup.org/udefinfo/defs.htm
Dr. Chris Harding – [email protected]
Ron Schuldt – [email protected], 720-842-6414