Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | rohan-wynder |
View: | 212 times |
Download: | 0 times |
©1998, 1999, 2000, 2001 Rational Software - All rights reserved©1998, 1999, 2000, 2001 Rational Software - All rights reserved
Session CM11Session CM11
Release Managementfor the UBS Data Warehouse Project
Release Managementfor the UBS Data Warehouse Project
Friedrich LehnUBS AG, Switzerland
Friedrich LehnUBS AG, Switzerland
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
AgendaAgenda
Project Overview Project Infrastructure Change Management Release Process Summary
Project Overview Project Infrastructure Change Management Release Process Summary
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project OverviewProject OverviewUBS
Global, integrated investment services firmand leading bank in Switzerland
World’s largest private bank Total client assets over US$ 1.5 trillion Acquired in 2000
UBS
Global, integrated investment services firmand leading bank in Switzerland
World’s largest private bank Total client assets over US$ 1.5 trillion Acquired in 2000
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project OverviewProject OverviewData Warehouse Program (DWP)
Establish common infrastructure for analytical data processing
Provide a business oriented set of data warehouseand data mart services
Standardized business data model Align the bank’s data mart portfolio Improve flexibility, time-to-market and data quality
Data Warehouse Program (DWP)
Establish common infrastructure for analytical data processing
Provide a business oriented set of data warehouseand data mart services
Standardized business data model Align the bank’s data mart portfolio Improve flexibility, time-to-market and data quality
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project OverviewProject OverviewData FlowData Flow
System of RecordsSystem of Records
Common Data / Business WarehouseCommon Data / Business Warehouse
Data MartsData Marts
extract, condition & loadextract, condition & load
data mart sourcingdata mart sourcing
visualizationvisualization
business userbusiness user
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project OverviewProject OverviewDelivery Streams
Main organizational element:team working on a subject data area / data mart
Three letter acronym as base for naming standards
Standardized infrastructure:UNIX directories, access group, meta data area, ...
Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager
Team size typically between 1 and 5
Delivery streams release independently from each other
Delivery Streams Main organizational element:
team working on a subject data area / data mart
Three letter acronym as base for naming standards
Standardized infrastructure:UNIX directories, access group, meta data area, ...
Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager
Team size typically between 1 and 5
Delivery streams release independently from each other
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project OverviewProject OverviewApplication StructureApplication Structure
<MAR> <MAR>...
RCL MDR FDS
System of Records
DSFDSF
Sourcing Framework
<SDA> <SDA>...
RCL release control toolsMDR meta data repositoryDSF DWP sourcing frameworkFDS feed configuration files<SDA> subject data area<MAR> data mart
RCL release control toolsMDR meta data repositoryDSF DWP sourcing frameworkFDS feed configuration files<SDA> subject data area<MAR> data mart
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project InfrastructureProject InfrastructureSystem EnvironmentsSystem Environments
• IBM AIX SP2 cluster• DB2 UDB EEE• PowerCenter V1.7• DWP Sourcing Framework• Cognos / Business Objects• ClearCase V3.2
• IBM AIX SP2 cluster• DB2 UDB EEE• PowerCenter V1.7• DWP Sourcing Framework• Cognos / Business Objects• ClearCase V3.2
Development ProductionTest
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project InfrastructureProject InfrastructureLogical Environments and Release StructureLogical Environments and Release Structure
Development Test Production
D A P delivery stream development
T framework development
framework test / delivery stream migration to new framework releases
X F V
E emergency releases
mandatoryoptional
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project InfrastructureProject InfrastructureRelease Cycles
delivery stream development: D A P
framework development: T X F Vafter sign-off: X D A P
migration to new frameworkreleases: [DAP] X F
emergency releases: [DAP] E P
Release Cycles
delivery stream development: D A P
framework development: T X F Vafter sign-off: X D A P
migration to new frameworkreleases: [DAP] X F
emergency releases: [DAP] E P
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project InfrastructureProject Infrastructure
Two areas:/dwp_root release area, version controlled/dwp_data dynamic data, archival on demand
Additional directory level in order to support more than one logical environment on one system
/dwp_root is organized by delivery streams, e. g.:/dwp_root/d/streams/rcl/bin
~<user>/dwp_root user specific development area
Two areas:/dwp_root release area, version controlled/dwp_data dynamic data, archival on demand
Additional directory level in order to support more than one logical environment on one system
/dwp_root is organized by delivery streams, e. g.:/dwp_root/d/streams/rcl/bin
~<user>/dwp_root user specific development area
Directory StructureDirectory Structure
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Project InfrastructureProject InfrastructureDirectory Structure (continued)
/dwp_data is organized by logical processing steps, e. g.:/dwp_data/p/data/landing (landing area)/dwp_data/p/data/tgtfiles (target files)/dwp_data/p/logs/system (framework log area)
Tool support for generation of directories in source environments (delta processing)
Automatic creation of missing directories in target environments by release procedures
Directory Structure (continued)
/dwp_data is organized by logical processing steps, e. g.:/dwp_data/p/data/landing (landing area)/dwp_data/p/data/tgtfiles (target files)/dwp_data/p/logs/system (framework log area)
Tool support for generation of directories in source environments (delta processing)
Automatic creation of missing directories in target environments by release procedures
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Change ManagementChange ManagementDesign Principles
Support different, clearly separated environments with different responsibilities
All environments have identical structure (products, databases, server configurations)
All program changes are done on the development system
All changes on test and production systems go through the release process and are clearly tracked
Design Principles
Support different, clearly separated environments with different responsibilities
All environments have identical structure (products, databases, server configurations)
All program changes are done on the development system
All changes on test and production systems go through the release process and are clearly tracked
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Change ManagementChange ManagementClearCase Set-up
One single VOB /vobs/dwp
Fully automated access layer (freeze and deliver routines)
Same directory structure as below /dwp_root, directories are automatically created
In general: only linear version trees (important: synchronization with database change management)
Branch support planned for emergency releases and migration path only
ClearCase Set-up
One single VOB /vobs/dwp
Fully automated access layer (freeze and deliver routines)
Same directory structure as below /dwp_root, directories are automatically created
In general: only linear version trees (important: synchronization with database change management)
Branch support planned for emergency releases and migration path only
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Change ManagementChange ManagementRelease Naming
<delivery stream>_<major>.<minor>.<patch>e. g.: RCL_1.1.0
<major> major release number(high level “wave” planning)
<minor> minor release number(delivery stream development plan)
<patch> patch level (bug fixes)
Emergency releases:RCL_1.1.0_sos_<#>
Release Naming <delivery stream>_<major>.<minor>.<patch>
e. g.: RCL_1.1.0
<major> major release number(high level “wave” planning)
<minor> minor release number(delivery stream development plan)
<patch> patch level (bug fixes)
Emergency releases:RCL_1.1.0_sos_<#>
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Change ManagementChange ManagementVersioning ExampleVersioning Example
RCL_1.0.0
RCL_1.0.1
RCL_1.0.2
RCL_2.0.0
RLS_P
RLS_A
common.pm
/main/1
/main/2
/main/4
/main/5
/main/3
RLS_A current version inuser acceptance test
RLS_P current version in production
/main/sos/1RCL_1.0.0_sos_1
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessChange Control Board
Responsible for high level planning and impact analysis
Defines release scope and release numbers on baseof delivery streams
Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible)
Result is documented in “wave plan” document
Change Control Board
Responsible for high level planning and impact analysis
Defines release scope and release numbers on baseof delivery streams
Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible)
Result is documented in “wave plan” document
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessRoles & ResponsibilitiesRoles & Responsibilities
developer development, unit and integration testing
delivery stream manager manager, release planning
database administrator database change control
release manager deployment, tracking, configuration control,
administration
RoleRole ResponsibilityResponsibility
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessProcess OverviewProcess Overview
Development ProductionTest
ClearCase
Freeze
Deliver
Receive
Deployment Package
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessRelease Objects
PowerCenter mappings
scripts / SQLs
Job dependency data
Uniserv jobs (address conversions)
Database objects (see below)
not included: documentation (intranet database)
Release Objects
PowerCenter mappings
scripts / SQLs
Job dependency data
Uniserv jobs (address conversions)
Database objects (see below)
not included: documentation (intranet database)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessDatabase Objects
tables (regular, summary)
views (business, security)
keys (unique, primary, foreign)
indexes
triggers
aliases
check constraints
not included: table spaces (system dependent)
Database Objects tables (regular, summary)
views (business, security)
keys (unique, primary, foreign)
indexes
triggers
aliases
check constraints
not included: table spaces (system dependent)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessRelease ProcedureRelease Procedure ResponsibleResponsible
1. Submit database change request *) delivery stream manager
2. Implement database changes *) database administrator
3. Prepare release area delivery stream manager (UNIX, PowerCenter, job dependencies, Uniserv)
4. Submit release request delivery stream manager
5. Prepare release area (DDLs) *) database administrator
6. Create new release (Freeze) release manager
7. Create deployment package (Deliver) release manager
8. Apply database changes to target system *) database administrator
9. Install release in daily deployment window release manager(Receive) (IT integration / IT operation)
*) in case of database changes only
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessFreeze ProcessFreeze Process
freeze <delivery stream> [ -patch | -minor | -major | -sos <release> ]
1. Retrieve previous release
2. Compare with release area (check for new, changed, deleted files)
3. Display results and ask for confirmation
4. Apply changes in ClearCase
5. Create and attach release label
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessDeliver ProcessDeliver Process
deliver <delivery stream> -t <target env.> [ -r <release> ] [ -a ]
1. Retrieve specified / latest release in ClearCase
2. Retrieve target environment file versions and create delta
3. Use -a(ll) for initialization / synchronization
4. Create deployment package (tar file + control file)
5. Update target labels
6. Lock release label
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReceive ProcessReceive Process
receive
1. Check hand-over area for pending releases
2. Remote copy deployment package to target system
3. Install it
4. Standardized post-installation steps: e. g. access permissions
5. Delivery stream defined post-installation steps (PostInstall.ksh file):e. g. for non-standard path names, generators, setuid bits
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReleaseRequest (1)ReleaseRequest (1)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReleaseRequest (2)ReleaseRequest (2)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReleaseRequest (3)ReleaseRequest (3)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessDatabase Change Management
PATROL DB-Change Manager by bmc software
Scope filter: assign database objects to delivery stream( view in ClearCase, name equal to delivery stream)
Apply database changes to development database
Create release baseline:freeze all database object versions for a delivery stream( label in ClearCase, name equal to release label)
Export DDL to release area(for documentation & change detection)
Database Change Management
PATROL DB-Change Manager by bmc software
Scope filter: assign database objects to delivery stream( view in ClearCase, name equal to delivery stream)
Apply database changes to development database
Create release baseline:freeze all database object versions for a delivery stream( label in ClearCase, name equal to release label)
Export DDL to release area(for documentation & change detection)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessDatabase Change Management (target system)
Target baseline: target database version(delivery stream plus timestamp as name)
create delta DDL depending on release baselineand latest target baseline (PATROL)
apply delta DDL
create new target baseline
Database Change Management (target system)
Target baseline: target database version(delivery stream plus timestamp as name)
create delta DDL depending on release baselineand latest target baseline (PATROL)
apply delta DDL
create new target baseline
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessDatabase Retrofitting Process
Rationale: certain database changes have to be applied and tested directly on the Production system (load performance optimization: indexes, summary tables, ...)
In order to include target system changes into next regular release, changes are promoted back to Development using the Retrofitting Process
In principle: new, database administrator driven release in ClearCase and PATROL that is not delivered
Database Retrofitting Process
Rationale: certain database changes have to be applied and tested directly on the Production system (load performance optimization: indexes, summary tables, ...)
In order to include target system changes into next regular release, changes are promoted back to Development using the Retrofitting Process
In principle: new, database administrator driven release in ClearCase and PATROL that is not delivered
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReleasing Meta Data
PowerCenter and JDM data are stored in the database
Uniserv jobs are stored in vendor specific directories
Unload utilities to export the corresponding dataand store it in the delivery stream´s release area (tar file)
Meta data is frozen and delivered together with all otherfile system objects
Load utility for loading objects in target environment
e. g.: meta_jdm <delivery stream> [ unload | load ]
Releasing Meta Data
PowerCenter and JDM data are stored in the database
Uniserv jobs are stored in vendor specific directories
Unload utilities to export the corresponding dataand store it in the delivery stream´s release area (tar file)
Meta data is frozen and delivered together with all otherfile system objects
Load utility for loading objects in target environment
e. g.: meta_jdm <delivery stream> [ unload | load ]
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessReleaseDatabaseReleaseDatabase
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Release ProcessRelease ProcessRelease Database(continued)Release Database(continued)
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
SummarySummaryExperiences
Over 1500 releases since May 2000
Effort for creation and installation of new release:5 - 60 minutes depending on amount of meta data(without database changes)
Main effort necessary for handling of meta data
No ClearCase problem encountered so far
Experiences
Over 1500 releases since May 2000
Effort for creation and installation of new release:5 - 60 minutes depending on amount of meta data(without database changes)
Main effort necessary for handling of meta data
No ClearCase problem encountered so far
Click on the desired colorClick on the desired color
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.
SummarySummaryWish List
Transaction concept for set of cleartool commands:What happens if the 199th check-in of 200 fails? Roll back?
Signal handling option for cleartool:Today it is not possible to ignore SIGINT in cleartool( manual clean up)
Wish List Transaction concept for set of cleartool commands:
What happens if the 199th check-in of 200 fails? Roll back?
Signal handling option for cleartool:Today it is not possible to ignore SIGINT in cleartool( manual clean up)
Questions?Questions?
Thank You!Thank You!
Friedrich H. [email protected]
www.fhlConsult.com
Friedrich H. [email protected]
www.fhlConsult.com
This presentation will be posted by tomorrow at: http://www.rational.com/ruc
This presentation will be posted by tomorrow at: http://www.rational.com/ruc