S i #S317222Session #S317222Application Change & ConfigurationManagement: Tales from the TrenchesJagan Athreya & Andy Oppenheim Mervyn LallyJagan Athreya & Andy OppenheimProduct ManagementOracle Corporation
Mervyn LallyGlobal IT Director, Business DevelopmentIngersoll Rand
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making a purchasing decision The development release and timing of anypurchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
© 2010 Oracle Corporation 333
Oracle Enterprise ManagerSession #S317222Session #S317222
Application Change & Configuration Management: g g gTales from the Trenches
Rolling out and maintaining enterprise applications inRolling out and maintaining enterprise applications in customer data centers can be highly complex tasks involving various parties ranging from IT security to application manager to database administrators. In this session, you willmanager to database administrators. In this session, you will learn about real world techniques from customers for managing change and ensuring compliance of your enterprise applications to ensure trouble-free and successful application change management tasks, such as application upgrades, IT security compliance.
© 2010 Oracle Corporation 444
Business-Driven IT Management
© 2010 Oracle Corporation 5
Session Agenda
<Insert Picture Here>• Admin challenges and app lifecycle g yscenarios
• Key configuration and change management featuresmanagement features
• Track application components• Standardize application configurationStandardize application configuration• Release and promote changes• Compliance• Customer Case Study – Ingersoll Rand
© 2010 Oracle Corporation 6
Application Configuration ComponentsEvolution of Configuration and Change Managemento u o o Co gu a o a d C a ge a age e
+ + + =A li tiSOA dD t bS d Application Software
ConfigurationApplication S h
SOA and Middleware
Configuration
DatabaseSoftware
Configuration
Server and Storage
Configuration
Application Configurationand Change ManagementSchema
Business Logic
Management
© 2010 Oracle Corporation 777
Current ChallengesSpan the Application Stack and the Application LifecycleSpa e pp ca o S ac a d e pp ca o ecyc e
• Administrative silos – Test DBA & d ti DBA SYSADMIN & DBAproduction DBA, SYSADMIN & DBA
– Limited visibility, finger-pointing
• Difficult to detect changes across
Applicat• Difficult to detect changes across
application deployment life cycle– Time spent in detection and debugging
tion Stack
• Manual processes for rolling out application changes
– Higher cost and time to delivery
• Fragmented reporting– Inaccurate compliance audits
© 2010 Oracle Corporation 888
Commonly Faced Scenarios
Spanning the Application Stack• The Employee Portal has performance issues – it was working last week, what changed?
• If I make these database upgrades will any of my applications be affected?• If I make these database upgrades, will any of my applications be affected?
• The admin made changes to the production web site configuration before leaving, what were they?
Spanning the Application LifecycleSpanning the Application Lifecycle• Test and Stage are configured the same, but I’m having issues when I migrated – why?
• How do I promote my configuration and code from Dev to Production?
• Wh do e ha e so man in alid schema objects in De b t not in Prod ction?• Why do we have so many invalid schema objects in Dev, but not in Production?
Spanning the Entire Data Center• How many E Business Suite versions 10/11 do I have that need to be upgraded?• How many E-Business Suite versions 10/11 do I have that need to be upgraded?
• How can I generate a SOX compliance report for all my Production and Stage applications?
• Support needs database info to debug the Application – where & how do I provide this?
© 2010 Oracle Corporation 999
Key Enterprise Manager FeaturesApplication Configuration and Change Management
View application dependencies hierarchically
Discover application assets via blueprint
pp ca o Co gu a o a d C a ge a age e
Track
TRACKING
Discover application assets via blueprint
Automate configuration upload to My Oracle Support for fast issue resolution
Automate drift checks:
Track Application
Components
STANDARDS
Between systems such as Production/Stage/Test or Primary/Standby
With saved gold standardsStandardize Application
Configuration
Automate promotion of application code from one deployment phase to another, e.g. Test to Production
Analyze impact of application upgrades on schema customizations
LIFECYCLE
y p pp pg
Automate detection of application schema changes
Detect configuration and schema changes in real time
Release and Promote Changes
COMPLIANCE
© 2010 Oracle Corporation 101010
Produce Rich Compliance reports involving multiple applications
Out-of-box mapping with SOX, PCI, etc.
Comply with IT Policies
Track
TRACKINGTrack Application ComponentsEnterprise Manager Features Track
Application Components
e p se a age ea u es
• Configurable UI providing customer flexibility to define
• Hierarchical view showing dependencies and assets
Features Benefits
customer flexibility to define how assets are viewed
• Blueprint-driven discovery and dependency mapping of
dependencies and assets, simplifying the way you manage your business
• Automated discovery that candependency mapping of configurations
• Automated configuration upload to My Oracle Support
• Automated discovery that can be extended for custom applications and eliminate manual trackingupload to My Oracle Support manual tracking
• Reduce problem resolution cycle by aiding Support
© 2010 Oracle Corporation 111111
Configurable UI, Define How Your Assets Are ViewedHierarchical View Showing Dependencies and
Define Your View
Assets Matching Your Business
• Customer Configurable UI– As many views as needed– Horizontal technology view– Application dependencies– Datacenter view– Cross datacenter view
© 2010 Oracle Corporation 121212
Blueprint Driven DiscoveryFlexibility and Easily Extended for Custom Applicationse b y a d as y e ded o Cus o pp ca o s
• Blueprints for Oracle and Third Party products (available via OTN)
• Support for all your application environments
Features Benefits
p ( )• Easily extended and customizable by
customer• Add additional collections for Files, DB
Objects and OS commands• Quickly support and maintain custom
applications
© 2010 Oracle Corporation 131313
pp
Blueprint Driven DiscoveryBlueprints for Oracle and Third Party Products (Available via OTN)
Out-Of-Box Oracle Technology Network
uep s o O ac e a d d a y oduc s ( a ab e a O )
• Extensive Blueprints out-of-box for Oracle and third party products• Updated and new Blueprints available for customer download from OTN
© 2010 Oracle Corporation 141414
Updated and new Blueprints available for customer download from OTN
Easily Extended and Customizable by CustomerAdd Additional Collections for Files, DB Objects and OS Commands, j
Add Additional Collections – Here’s How
• Select an existing Blueprint that you want to modify • Point to an existing Host and Base Directory
N fi ti fil t b dd d i l t
© 2010 Oracle Corporation 151515
• New configuration file to be added is sqlnet.ora
Easily Extended and Customizable by CustomerAdd Additional Collections for Files, DB Objects and OS Commands
Add Additional Collections
, j
Bl i t i d ill di l th il bl fil• Blueprint wizard will display the available files• Select the desired file to be added to the configuration
collection (sqlnet.ora)
© 2010 Oracle Corporation 161616
( q )
Easily Extended and Customizable by CustomerAdd Additional Collections for Files, DB Objects and OS Commands
Add Additional Collections
, j
• sqlnet.ora now added to database configuration collectionD !
© 2010 Oracle Corporation 171717
• Done!
Easily Extended and Customizable by CustomerAdd Additional Collections for Files, DB Objects and OS Commands
Adding a New Blueprint
, j
• Select Add New Resource Specification
© 2010 Oracle Corporation 181818
Easily Extended and Customizable by CustomerAdd Additional Collections for Files, DB Objects and OS Commands
Adding a New Blueprint
, j
• Select name for new Blueprint• Follow same process for adding a new collection
© 2010 Oracle Corporation 191919
p g
Automated Configuration Upload to My Oracle SupportReduce Problem Resolution Cycle by Aiding Support
My Oracle Support
My Oracle Support – Enterprise Manager: Unified View, Integrated Information
y y g pp
My Oracle Support
Problem/SRManagement
ConfigurationManagement
KnowledgeManagement
KnowledgeM t
Problem/SRM t
ConfigurationM t
Performance Management
Provisioning & Patching
Problem DiagnosisC t
Management Management Management
Management PatchingDiagnosisCustomer
DatabasesHardware Middleware Applications
© 2010 Oracle Corporation 2020
Automated Configuration Upload to My Oracle SupportReduce Problem Resolution Cycle by Aiding Supporty y g pp
• Unified Support and Management View• Unified Support and Management View– Patches and Updates– Knowledge– Service Requests
© 2010 Oracle Corporation 212121
Service Requests
Automated Configuration Upload to My Oracle SupportReduce Problem Resolution Cycle by Aiding Supporty y g pp
• Increase accuracy Service Request auto populated• Increase accuracy - Service Request auto populated• Decrease resolution time – configuration information associated
with Service Request
© 2010 Oracle Corporation 222222
Standardize Application ConfigurationsEnterprise Manager Features
STANDARDS
e p se a age ea u esStandardize Application
Configuration
• Automated drift checking • Quickly identify differences
Features Benefits
gacross application Life Cycle (Dev Prod)
• Define Gold Standards and/or
y ythat are impacting your operations
• Established standards across Baselines
• Automated notification of configuration drifts
your Datacenter• Proactive notification prior to
issues occurring in your operation
© 2010 Oracle Corporation 232323
Automated Configuration Drift CheckingQuickly Identify Differences Impacting OperationsQu c y de y e e ces pac g Ope a o s
WebApplications
MidTiers
DatabaseBusiness
Applications
Database
• Complex configuration comparison
• Compare entire application stacks or technology layers
Features Benefits
• Compare one-to-one, one-to-many
• Save comparisons and versions
• Compare all components of your operation
• Understand what has changed over time
© 2010 Oracle Corporation 242424
over time
Complex Configuration CompareCompare Entire Application Stacks or Technology LayersCo pa e e pp ca o S ac s o ec o ogy aye s
• Compare complete applications stacks across:– Deployment lifecycle– Datacenters
© 2010 Oracle Corporation 252525
Datacenters
Complex Configuration CompareCompare Entire Application Stacks or Technology LayersCo pa e e pp ca o S ac s o ec o ogy aye s
• Only differences are shown, and highlighted in blue• Ignores specified properties in subsequent compares• Easily find the “needle in the hay stack”
© 2010 Oracle Corporation 262626
y y
Save Comparisons and VersionsUnderstand What Has Changed Over TimeU de s a d a as C a ged O e e
• All configuration collections are save and versioned• Saved versions can be compared against current• Save versions and be compared to other resources
© 2010 Oracle Corporation 272727
Save versions and be compared to other resources
Define Gold Standards and/or BaselinesEstablished Standards Across Your Datacenters ab s ed S a da ds c oss ou a ace e
Here’s how to define a standard for:– Technology Stack
Technology Component– Technology Component
• Right click the item you want to define as your standard
• Give it a name and “save as template”
• Standard is saved under Templates
• Utilize saved standard for ad-hoc or scheduled compares
© 2010 Oracle Corporation 282828
compares
Automated Notification of Configuration ChangesProactive Notification Prior to Issues Occurring in Your Operationoac e o ca o o o ssues Occu g ou Ope a o
• Perform a comparison• Save the comparison (disk icon upper right)• Name the comparison for future reference
© 2010 Oracle Corporation 292929
Name the comparison for future reference
Automated Notification of Configuration ChangesProactive Notification Prior to Issues Occurring in Your Operationoac e o ca o o o ssues Occu g ou Ope a o
• Locate your saved comparison under Systems, Comparisons• Right click on your saved comparison• Enter schedule and notification information
© 2010 Oracle Corporation 303030
Release and Promote ChangesEnterprise Manager Features
LIFECYCLE
e p se a age ea u es
Release and Promote Changes
• Impact analysis of application • Eliminate upgrade utilities and
Features Benefits
p y ppupgrades on customizations
• Promote application code and schema changes across
pgtesting of customizations
• Reduce time and eliminate errors of application g
deployment phases (Test Prod)
ppdeployments or migrations
© 2010 Oracle Corporation 313131
The Problem: Vendor Releases Application UpgradesApplication Upgrades
StepsC t B li b dUpgrades
&Patches
Version 11 5 10 Version 12
• Create Baseline based on dependencies
• Upgrade applicationCompare upgraded
What has changed• TABLES
What is impacted• Data loader programs
Version 11.5.10 Version 12 • Compare upgraded application schema with baseline
• Identify changes• TABLES• INDEXES• PACKAGES• PROCEDURES
• Data loader programs• Application
customizations• Integration with other
Identify changes– Upgrade
customization modules if affected.
• MATERIALED VIEWS• TABLESPACES• init.ora
Integration with other applications – Do NOTHING if
unaffected
© 2010 Oracle Corporation 3232
Dependency Mapping Accelerate Planningcce e a e a g
• CustomizationI i Ch k P i ti M d l
• Dependent ObjectsPO VENDORS t bl– Invoice Check Printing Module – PO_VENDORS table
– AP_CHECKS_PKG package
© 2010 Oracle Corporation 3333
Comparison of Dependencies Before and After the Application Upgradee o e a d e e pp ca o Upg ade
© 2010 Oracle Corporation 3434
Promote Application Code Across Deployment Phases
Dictionary Baseline– All schema objects for an application– Point-in-time, versioned snapshot stored in the EM repository
HREMP DPT
OE , p p y– Baseline scope
Saving Development Changes in Baseline Versions– Automated: creates new versions for new sets of planned
ORDSALE
Automated: creates new versions for new sets of planned changes
– Fault tolerant: saves changes outside the database in EM repository
v1 v2 v3
– Auditable: preserves history of planned changesPropagation of Planned Changes – 4 Step Process
1. Validate planned changes to identify conflicts or previous changesv3
2. Preview and edit validated changes before applying3. Generate SQL script of final set of validated changes4. Apply validated planned changes
© 2010 Oracle Corporation 3535
pp y p g
Promote Application Code Across Deployment Phases Change Propagation vs. SQL Scripts
Category Change Propagation SQL ScriptsIntelligent validation of h
Yes No
Change Propagation vs. SQL Scripts
changes
Preview and edit changes before apply
Yes No
Automatically save and version Yes Nochanges
Display log of changes applied Yes YesMaintain history of change logs Yes No1
C t ll d Y NCentrally managed Yes No
Allows execution without revealing database passwords
Yes No
Designed to handle database Yes Nochanges across multiple database environments
1Can be handled by manual processes
© 2010 Oracle Corporation 3636
Comply with
COMPLIANCEComply With IT PoliciesEnterprise Manager Features Comply with
IT Policiese p se a age ea u es
Features Benefits
• Automated detection of application schema changes
• Eliminate the effects of unplanned changes
Features Benefits
• Real-time configuration change detection
• Out-of-the-box frameworks
• Detect who, when and what was changed at the point of occurrence
mapping to SOX, PCI, …• Automated compliance
reporting across your
• Faster rollout of compliance polices
• Reduce time and effort in applications meeting your audit needs
© 2010 Oracle Corporation 373737
Automated Detection of Application Schema Changes Production Complianceoduc o Co p a ce
• Identify out-of-band changes to production database SQLproduction database– Best practice: Weekly comparison report
of current production against gold master baseline of production schema
SQL
– Eliminate the effects of unplanned changes• Malicious SQL
SQL
• Unauthorized Access Control changes• Scope
– Index– Tables– PL/SQL objects
© 2010 Oracle Corporation 383838
Change Detection: DBA 1.0 vs 2.0Scenario: Compare Test and Production SchemaScenario: Compare Test and Production Schema
1 Identify databases test and production 1 Run Dictionary Comparison job
DBA 1.0 DBA 2.0
1. Identify databases, test and production2. Get access to application schema3. Write script to get table column parameters4. Run script on first database
1. Run Dictionary Comparison job2. Identify all schema and
initialization parameters differences in report
4. Run script on first database5. Run script on second database6. Visually compare results7. Write new script for indexes8. Re-run script9. Repeat for all other database objects10.Generate a report listing changes11.Write script to apply changes12.Apply changes13.Repeat above steps to verify changes14 R thi f t i i d d ti
© 2010 Oracle Corporation 3939
14.Re-run this for training and production15.Re-run this for development and production
Real-Time Configuration Change Detection Detect Who, When and What Was ChangedReconciliation with Change Management SystemsDetailed Change Detection, Files, DB Objects, User Processese ec o, e a d a as C a ged
• Reconciliation with Change Management Systems
• Change detection for Files DB
g g yg , , j ,
• Change detection for Files, DB Objects, Users Processes
• Identify authorized and unauthorized changes
• Extensible across application environment
• High level dashboard for your S it tSecurity teams
• Out-of-the-box frameworks• Quickly identify areas of
concern and detailed drill downconcern and detailed drill down
• Detected changes reconciled with Change Management Systems• Determine authorized and unauthorized changes
D ill d i t f th d t il if d d
© 2010 Oracle Corporation 404040
• Drill down into further detail if needed
Out-of-box Frameworks Mapping to SOX, PCI, … Flexible Framework for Compliance Policies and Ease of Use
Configuration Change ConsoleEnterprise Manager
e b e a e o o Co p a ce o c es a d ase o Use
• Configuration Change Console – Real-time change detection– Out-of-box compliance frameworks and customer extensible
• Enterprise Manger – Security, Configuration, Storage Policies– Out-of-the-box mapping and customer extensible
© 2010 Oracle Corporation 414141
Automated Compliance Reporting Across ApplicationsReduce Time and Effort in Meeting Your Audit Needs
Out-of-box Compliance ReportsPolicy and Best Practice Reporting
g
• Configuration Change Consoleg g– Out-of-box reports focused on meeting your audit requirements
• Enterprise Manager– Reporting of policies and best practices across your datacenter
© 2010 Oracle Corporation 424242
Reporting of policies and best practices across your datacenter
Mervyn Lally – Ingersoll Rand, Enterprise Services
Driving a Multi-Year IT Project for Consolidation, Standardization and ERP Deploymentp y
Questions
© 2010 Oracle Corporation 444444
© 2010 Oracle Corporation 45