of 29
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
1/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Oracle EPM Troubleshooting& Infrastructure Best
Practices
Doug FiedlerPrincipal Infrastructure Architect
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
2/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Focus
ServicesPeople
Methodology
Customers
Partnership
15 Years
700+ clients
1000+ projects
About Edgewater Ranzal
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
3/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
We offer a full spectrum of EPM/BI ServicesDashboards & Scorecards, Financial Analytics& Reporting, Operational Analytics, What-if
Analysis, Query & Reporting, Visual ExplorationFinancial performance, Legal,Segment & Mgmt Reporting, Financial Close
HFM Optimization, Performance LabSOX Compliance Support
Strategic Finance, Planning, Budgeting,Forecasting, Workforce Planning,Capital Planning, Project FinancialPlanning
Data Integration, Financial Data
Management, Data Warehousing,Master Data Management &DRM,
ETL Services, Automation
Project/Program Mgmt, EPMRoad Maps, ApplicationReviews, BusinessRequirements, ProcessChange, Documentation
Installation, Upgrades,Migration, System
Monitoring, Backup andRecovery, Disaster Recovery,
Load Testing, HardwareSizing, Exalytics
Benchmarking
Consolidation
BusinessIntelligence
EnterprisePlanning
Infrastructure
Training &Support Services
ProjectManagement
DataServices
Costing &Profitability
Mgmt
Support Services Infrastructure &Application Support Contracts
Key Teach Course Delivery: Planning, Essbase,Financial Reporting, Smart View, HPCM, HFM, FDM,
DRM, OBIEECustom Training Delivery: Process & Reporting
HPCM Standard & DetailedModels, Waterfall Allocations,Activity Based Costing, Customer,Product & LOB Profitability
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
4/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
v11.1.2.3
V11.1.2.2
V11.1.2.1
V11.1.1.4/3 or earlier
What version of EPM are you running today?
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
5/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Monitoring your EPM environment with OEM
EPM log utilitiesziplogs, log analysis, validation
ResetConfigTask - revert a product back to an un-configured state
Maintenance & Housekeeping - common logs to
rotate/archive
When to apply PSUs
Virtualizationwhen it works and when it doesnt
Agenda
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
6/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
What is OEM? OEM stands for Oracle Enterprise Manager,it is used to manage many Oracle products includingOracle RDBMS, OBIEE & EPM. It can be used to monitorperformance, monitor system health, restart services andmany other useful features.
Out of the box it only plugs into the weblogic JVMs but itis possible to add non JVM applications like Essbase.
Upgrade to OEM 12c and check out doc -http://www.oracle.com/technetwork/middleware/bi-foundation/epm11123-oem12c-monitor-1955582.pdf
Monitoring your EPM environment with OEM
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
7/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
To get to the OEM login
screen you must first start the
weblogic admin server inforeground
This can be added as a
windows service -
https://support.oracle.com/ep
mos/faces/DocumentDisplay?
id=1060058.1
https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.1https://support.oracle.com/epmos/faces/DocumentDisplay?id=1060058.18/10/2019 DFiedler.OracleEPMTroubleshooting (2)
8/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
Here you have access to all
weblogic deployments on the
current wls domain
Green for jvms that are up, red for
down
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
9/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
Shows system health
JVM Performance
Response & Load times
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
10/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
Start/stop services View logs
Setup security
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
11/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
Log Analysis Utility
Located in
\Oracle\Middleware\user_projects\epmsystem1\bin\loganalysis.bat
Collects data from all log EPM log files, allowssearching by key word and/or window of time
EPM log utilities
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
12/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
\Oracle\Middleware\user_projects\epmsystem1\diagnostics\
reports\LogAnalysis_Report_2013_10_24_17_58_15.html
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
13/29
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
14/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
\Oracle\Middleware\user_projects\epmsystem1\bin\validate.bat
Checks to make sure that the environment is workingproperly and produces the following report
Validation Tool
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
15/29Copyright 2014 Doug Fiedler, Edgewater Ranzal
\Oracle\Middleware\user_projects\epmsystem1\
bin\registry-cleanup.bat
If you see errors pertaining to the EPM registry,after viewing the validation report, they can be
fixed by running the registry-cleanup tool.
It is also helpful to remove a server from your
environment in order to clean it out of the
registry.
Registry-cleanup
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
16/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Provides information on what has been configured, # of
CPUs/Ram by server, OS, Database/JDBC etc.
Deployment Report
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
17/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Generates an html report of all registry keys
Registry Report
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
18/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
What does it do?
It allows you to redo a config task
Why would you want to do this?
When a task fails or you need to changesomething
First run a registry report D:\Oracle\Middleware\user_projects\epmsystem1\
bin\epmsys_registry.bat or epmsys_registry.sh
ResetConfigTask
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
19/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Here we can see that Planning has beenconfigured and it was successful
If it had failed it would say failed instead ofconfigured
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
20/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
To reconfigure run the following command:
resetConfigTask.bat -product Planning -task
applicationServerDeployment
When you go back into the configtool you will now see
that the Planning deployment will have gone from green
to yellow (unconfigured)
You can do this with any epm configuration task
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
21/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Backups - Nightly File system (d:\oracle , /app/oracle)
Database (SQL, Oracle)
LCM exports
Log file archiving Essbase.log & app logs can be archived with the
AGTMAXLOGFILESIZE & APPMAXLOGFILESIZE commands within
the essbase.cfg HFM & .odl (oracle diagnostic logging) log files should be archived
manually as needed when they reach a size that is too big to open
with windows tools
If you have a log essbase.log file that cant be opened in windows you
can view it in EAS and view by date/time as well
Maintenance & Housekeeping
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
22/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
HFM Purge task & data audit quarterly
Miscellaneous Always monitor disk space, especially on the Essbase server. If the
disk becomes full Essbase will become corrupted.
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
23/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
PSUs tend to come out quarterly
PSU = Patch Set Update
PSE = Patch Set Exception
PSEs come out all the time, they are usually
created because of defects specific to a certain
customers environment I recommend applying PSUs quarterly
When to apply Patches
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
24/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
I only recommend applying PSEs if the issue
listed in the defects fixed readme is an issue
that you are experiencing To find out what the latest EPM patches are go
to the following url:
https://supporthtml.oracle.com/epmos/faces/ui/km/DocumentDisplay.jspx?id=1400559.1
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
25/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Oracles official policy on Virtualization
The following KB article is Oracles official stance on virtualizing the
Oracle EPM Product stack.
https://support.oracle.com/CSP/main/article?cmd=show&type=NO
T&doctype=HOWTO&id=588303.1
Additional information
https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ct
rl-state=1854yc0wrq_9&_afrLoop=14354592882224
In a Nut Shell
Oracle will support VMWare and Microsofts Hyper-V technologies,but they do not certify them. What this means is that if Oracle
cannot fix an issue they can refer you to the hosting provider for
further troubleshooting, or force you to physicalize the servers and
reproduce the issue. However, Oracle does certify their products
running on Oracles Virtualization Manager (OVM).
Virtualization
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
26/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Advantages Reduced hardware costs (ability to consolidate servers)
Much quicker to stand up a VM vs a physical server
If the hypervisor fails, technologies like vmotion allow you to moveyour VM/s to another host, this is also very helpful in DR situations
CPU, disk & memory can be added and removed as needed
Disadvantages
You can over assign resources (CPU, memory) and this can causeperformance issues when multiple VM guests need the resources at
the same time
Certain products run better than others when virtualized: weblogic
JVMs run well while Essbase & HFM may have a performance loss
during intense calculations
Virtualization
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
27/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Allocate resources statically : RAM and vCPUs
Dont let the VM Host Servers manage resources on
the fly, use reservations Dont over subscribe your Virtual host servers
Utilize dedicated VM Host servers for EPM when
possible
Test, Test, Test and then Test again. If you have a testtool like Load Runner ever better
Have performance issues? We have VMware Certified
Engineers who can find & fix your VM bottlenecks
VirtualizationWhats important to know
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
28/29
Copyright 2014 Doug Fiedler, Edgewater Ranzal
Physical Depending on the
application, the followingservers should perform
better on Physical
hardware
HFM application Server
(Consolidations) Essbase Server
(Calculations)
Integration Server (Data
Mapping and Loads)
Virtual Foundation and Web
Servers DRM Server
HSF Server
All servers can be virtual,
really depends on the
application
HFM Application Server
Essbase Server
Integration Server
8/10/2019 DFiedler.OracleEPMTroubleshooting (2)
29/29
Questions?