+ All Categories
Home > Documents > Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program...

Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program...

Date post: 16-Oct-2020
Category:
Upload: others
View: 7 times
Download: 0 times
Share this document with a friend
78
Technical Workshops | Esri International User Conference San Diego, California Administering your Oracle Geodatabase Travis Val Jim McAbee July 25, 2012
Transcript
Page 1: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Technical Workshops |

Esri International User Conference San Diego, California

Administering your Oracle Geodatabase

Travis Val Jim McAbee

July 25, 2012

Page 2: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Agenda

• Administration Topics - System Requirements and Configuration - 10.1 Administration Options - 10.1 Client and Connection Differences - Installation, Migration and Upgrade Scenarios - Managing Performance in 11g - Some Notes on Advanced Configurations - Loading, Backup and Spatial Types

• Pre-Requisites - Administration of Geodatabase Intro - Basic knowledge of database terminology - Working knowledge of ArcSDE Technology Basics

Page 3: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

System Requirements and Configuration

Page 4: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Major Themes 9.x/10.0 to 10.1

• System Requirements - 9.3/10 to 10.1 biggest impact on client configuration 32 vs. 64 bit,

instant client, etc…

• Administration Automation - Command Line Tools: Oracle runs on Unix/Linux so they may be

used – ArcSDE command line tools are separate install - 10.1 GDB Administrative interface and new GP tools

• Upgrade/Migration • Datatypes

(should be migrating out of LONG RAW to other types) • Oracle issues

- Datapump imports into 11.2.0.2 or 11.2.0.3, 11.2.0.1 tends not to have issue (GDB_Items, etc..) – ORA-00600 possible

Page 5: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

ArcGIS and Oracle Versions

ArcGIS Version

Oracle Versions

10.1 11.2.0.1 11.1.0.7 64-bit only 10.2.0.3

10.0 11.2.0.1 (64 bit) 11.1.0.7 (32 & 64) 10.2.0.3 (32 & 64)

9.3/9.3.1 11.2.0.1 (64 bit) 11.1.0.7 (64 bit) 11.1.0.6 (32 & 64) 10.2.0.3 (32 & 64) 9.2.0.7 (32 &64)

• Oracle Editions • Enterprise - Standard One – Standard

• ArcGIS Version Backwards Compatibility

9.3

10.1

9.1

10.0

9.2

Page 6: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

System Requirements: OS for Oracle 11g r2 • Check Operating System also…….including patch level requirements

10

10.1

Page 7: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

DBMS_PIPE, DBMS_LOCK, and other packages • What are they?

- DBMS_PIPE: allows sessions to communicate through memory - DBMS_LOCK: Oracle’s api to create locks in DB from SQL

• For ArcSDE granted execute to public

• Remove privileges from public? - Tech Article #35408 – Can the public privilege be removed from

Oracle’s dbms_pipe and dbms_lock packages?

• GRANT EXECUTE ON dbms_pipe TO public; • GRANT EXECUTE ON dbms_lock TO public; • GRANT EXECUTE ON dbms_lob TO public; • GRANT EXECUTE ON dbms_utility TO public; • GRANT EXECUTE ON dbms_sql TO public; • GRANT EXECUTE ON utl_raw TO public; • GRANT EXECUTE ON dbms_crypto TO sde;

Page 8: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle configuration parameters

• Oracle Memory: - System Global Area (SGA) and Program Global Area (PGA) - Memory areas used for - Interpreting SQL statements, Fetching

data and Submitting edits to database • Use Automatic Shared Memory Management (ASMM)

- Set SGA_TARGET through Oracle Enterprise Manager (OEM) • Cursor Configuration:

- Cursor: Points to rows returned by a SQL statement - Used to avoid re-parsing similar statements with bind variables - Stored in PGA and used extensively by ArcSDE - open_cursors = 2000 (or larger to avoid ORA-1000 error) - Tech Article #27024 – What is an appropriate value for the

Oracle init.ora parameter ‘open_cursors’

Page 9: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Controlling Storage • Tablespace Management based

on - Backup requirements -

Activity - Size of segment • SDE.DBTUNE table

- Specifies configuration keyword, storage type, storage parameters

• To create a keyword: - Export DBTUNE table to a file,

edit it and import back sdedbtune –o alter

Page 10: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

10.1 Clients and Connections

Page 11: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

ArcGIS Geodata Management technology stack

ArcSDE (ArcSDE libraries & stored procedures)

Oracle (RDBMS)

Geodatabase

Spatially enabled database

Database

Behaviors & Complex Features

Versioning & Distributed Data

Transaction Management, Spatial Types, Indexing

ArcGIS clients (ArcObjects/API’s)

Page 12: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Connection Architectures

ArcSDE libraries Geodatabase

gsrvr

giomgr

Oracle Client

Geodatabase

gsrvr Geodatabase Oracle Client

ArcSDE Libraries Database Server

giomgr

ArcSDE Libraries Database Server

“Direct Connect” – Recommended Method

“Application Server - standalone”

“Application Server”

1521

1521

5151

5151

Page 13: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

10.1 and Oracle Clients

• Administrator • Runtime • Instant (now supported at 10.1) • Remember ArcGIS Server now

64 bit at 10.1

ArcGIS for Desktop

32

ArcGIS for Server

64

64 bit Oracle

64 bit Oracle

32 bit Oracle Client

64 bit Oracle Client

Page 14: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

10.1 Connection Functionality

Geodatabase Database

10.0 Clients Query Layers introduced @ 10.0

10.1 Clients

Spatial Data ST_Geometry* SDO_Geometry

Geodatabase Administrative

Schema

*ST_Geometry SQL function support only available for supported Operating Systems (where library compiled)

10 10.1

Page 15: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Connecting at 10.1

10

10.1

Page 16: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

10.1 Oracle Instance Connection string

server name/service name (or ID) dbsrvr/orcl

IP address of server/service name (or ID) 10:10:10:10/orcl

server name:port/service name (or ID) dbsrvr:60000/orcl

IP address of server:port/service name (or ID)

10:10:10:10:60000/orcl or [6543:e04:0:1:f587:1249:12f9:w3ud]:60000/orcl

URL of server //dbsrvr:60000/orcl

server name only (if listener configured to point to default instance)

If instance not listening on default port number, must use connection syntax that includes port number.

Page 17: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Demo: Setting Up Instant Client

Page 18: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

10.1 Administration

Page 19: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Administrative and Maintenance Task more capabilities from desktop for GIS manager and staff

10.1 10

• Administrative task automation – more options at 10.1 - more functionality in Desktop and Server - SDE command line tools separate install & 64 bit (need 64 bit client)

Page 20: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Inclusion of Geodatabase Toolset

Page 21: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data
Page 22: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Multiple Geodatabase Configuration • Multiple Geodatabase for Different Tasks

- At least two Geodatabases should be present typically – production and test

- Other reasons for multiple geodatabases - Editing and publishing (web) - Production and test/development - Different application needs - Separate versioning management

- Methods for configuring multiple Geodatabases in Oracle

- Multiple instances on same server - Multiple instances on different servers - Project/Schema Geodatabase

Test/Dev. Publishing Production

Page 23: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

• Multiple Oracle Geodatabases – multiple instances or within same instance

• Geodatabase contained in specific User Schema • SDE “master” geodatabase present with Schema based

Geodatabase in instances table • Rules

- a user can own only one geodatabase - that user is the geodatabase administrator and requires those

database privileges - that user can only own data in that geodatabase - users can only own data in one geodatabase

Oracle Schema Geodatabase Admin An option for multiple Oracle Geodatabases

Page 24: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Notes on Upgrades and Migration

Page 25: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Administration Topics : Planning for New Configuration

• What are some of the more advanced issues? - Long Raw to BLOB - Geodatabase Administrative schema changes - Differences between Oracle versions

• Default Oracle settings are a good starting point except: - Database Cache / Memory, Max number of cursors a session can use,

Data Storage

• Estimate or measure load and storage: - Need to define a sample data, MXD and operations when possible - Oracle Enterprise Manager (OEM), Oracle Remote Diagnostic Agent

(RDA)

Page 26: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Upgrade Considerations

• Always perform any upgrades in a test environment • Backup any custom files in %SDEHOME%\etc directory

- dbinit.sde - dbtune.sde

• Upgrade OS and Oracle first, if needed, then test • Upgrade ArcSDE, then test • Backup, Backup, Backup!

- It is NOT a Backup unless it has been tested to make sure it works.

• Check existing GDB for any errors or redundancies • Clean DBMS_PIPE - Values in the database pipe can cause

connection problems • Upgrade to 10.x requires use of ArcObjects GP tool

- New Geodatabase Administrative schema changes

Page 27: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Upgrading Schema based Geodatabase

• Must upgrade master first and not simultaneously

• Must make a backup of the entire database; creating a backup of only a user's schema does not include the user-defined types and functions

• Stopping the ArcSDE service on the master geodatabase disables service connections to user-schema geodatabases

Page 28: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Migration Tool • LONG RAW data type being deprecated at Oracle 11g • Migrate Storage GP tool

- Easily convert GIS datasets from a different storage format to ST_GEOMETRY in Oracle

- Changes data “in place”, no new datasets are created - Must be the data owner to execute - Supports versioned & archived datasets

Page 29: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Upgrade Geodatabase – from ArcCatalog

Page 30: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

# Process: Upgrade Geodatabase arcpy.UpgradeGDB_management(<Connection file>, "PREREQUISITE_CHECK", "UPGRADE")

Upgrade Geodatabase

• Requires sde user upgrade permissions • Requires a direct connection to geodatabase • Pre-requisite check determines if geodatabase is

upgradable - See “Preparing to upgrade a geodatabase in Oracle”

topic for a full list of requirements

Page 31: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Installation, Migration, Upgrade Summary

• Start with recommended minimum parameters for New and monitor Existing

• Cursors and Memory (OPEN_CURSORS, SGA, etc..)

• Check connections parameter in server_config table via sdeconfig export/import command or sql

• Monitor usage, or ask DBA’s to monitor and adjust as necessary

• Establish a performance baseline so future growth and changes can be measured

• OEM, mxdperfstat, Windows performance monitor, ASH

Page 32: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

ESRI KB and Help Articles for Parameters/Upgrade

• FAQ: What is an appropriate value for the Oracle init.ora parameter 'open_cursors'? http://resources.arcgis.com/content/kbase?fa=articleShow&d=27024

• Error: ORA-01000: maximum open cursors exceeded http://resources.arcgis.com/content/kbase?fa=articleShow&d=28861

• HowTo: Identify a cursor leak in Oracle http://resources.arcgis.com/content/kbase?fa=articleShow&d=35090

• SQL Statements using st_geometry operators experience decreased performance in Oracle 11g http://resources.arcgis.com/content/kbase?fa=articleShow&d=35236

• Oracle Parameters help topics http://webhelp.esri.com/arcgisserver/9.3/java/index.htm#geodatabases/oracle_1010194088.htm

Page 33: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Spatial Types

Page 34: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

ESRI Spatial Type – ST_GEOMETRY

• User Defined Type (UDT) used to store geographic features • Allows access to spatial data through SQL functions • Efficiency - Automatic geometry validation • Conforms to ISO and OGC standards • Available since ArcSDE 9.2

Page 35: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Using SQL with ST_GEOMETRY

• Needs an external dll • Extproc needs to be set up for

- Executing SQL commands on a ST_GEOMETRY column - For using ST_GEOMETRY operators - Definition and label queries inside ArcMap

• With SQL and ST you can - Create tables with a spatial attribute - Read and Analyze the spatial data - Insert, update and delete simple geometry data

• Geodatabase behavior not supported through SQL

Page 36: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Spatial Types and Functions

• Creation of Features - through SQL

• Analysis - Buffering Geometry - Convex Hull - Difference of Geometries - Intersection of Geometries - Symmetric Differences - Union - Minimum Distance - Aggregates

• Feature Information

Page 37: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

• st_shapelib library • extproc.ora (11g) or listener.ora/tnsnames.ora

configuration

ST_Geometry spatial type configuration

Page 38: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Setting up the ExtProc via Listener and Tnsnames tnsnames.ora file:

listener.ora file:

EXTPROC_CONNECTION_DATA = (DESCRIPTION = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) (CONNECT_DATA = (SID = PLSExtProc) (PRESENTATION = RO)) )

LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST=AKLAP.esri.com)(PORT=1521)) ) )

LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) (ADDRESS = (PROTOCOL = TCP)(HOST=AKLAP.esri.com)(PORT=1521)) ) )

New Entry

Original Entry

Key must match

Page 39: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Setting up the ExtProc (10g)

LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) (ADDRESS = (PROTOCOL = TCP)(HOST=AKLAP.esri.com)(PORT=1521)) ) ) SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (SID_NAME = PLSExtProc) (ORACLE_HOME = C:\oracle\product\10.2.0\db_1) (PROGRAM = extproc) (ENVS="EXTPROC_DLLS=C:\Program Files\ArcGIS\ArcSDE\ ora10gexe\bin\st_shapelib.dll") ) )

listener.ora file:

Page 40: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

extproc.ora @ 11g

• Located in ORACLE_HOME\hs\admin directory

• Windows EXTPROC_DLLS=ONLY:C:\\mylibraries\\st_shapelib.dll

• Unix/Linux SET EXTPROC_DLLS=ONLY:/user/esrilibs/libst_shapelib.so

Page 41: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle ExtProc - for St_geometry

• Modify the listener.ora and tnsnames.ora files - Oracle Documentation - Webhelp topic - Configuring the Oracle listener to access the

geodatabase with SQL - Technical articles:

- 32188 - Error: ORA-28595: Extproc agent : Invalid DLL Path - 33003 - Error: ORA-06522: libsg.so: cannot open shared object

file: No such file or directory - 33004 - Bug: ORA-06522: (path)\st_shapelib.dll: cannot open

shared object file: No such file or directory - 38043 – Bug: Configure ST_GEOMETRY for use in Oracle 11G

Release 2 on Windows - 39119 – How to: Check if the Oracle extproc is set up correctly for

direct ST_GEOMETRY SQL queries

Page 42: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Demo: Configuring ST_Geometry

Page 43: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

• Set following in SDE.DBTUNE table when working with data stored in SDO_Geometry

- UNICODE_STRING set to “FALSE” (if VARCHAR vs. NVARCHAR is desired)

- GEOMETRY_STORAGE set to “SDO_Geometry” - SDO_SRID (if same SRID is always used) - other parameters

Few notes on SDO_Geometry

Page 44: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Managing Performance and Troubleshooting

Page 45: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Managing Performance in 11g : Statistics

• Table statistics - The distribution and contents of rows - What the optimizer uses to make execution plans

• Index statistics - Information about the rows stored in IOTs, and

other index metadata

• System statistics

- Internal object statistics

Page 46: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle table statistics • Describe segment characteristics for optimizer and DBA

- Critical for maintaining performance - Current statistics increase Oracle IO performance

• Update when data changes significantly: - Loading - Major editing

• Analyze through ArcGIS

Create statistics on attributes, geometry, versioning delta tables and historical archive

Page 47: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle 11g – Automatic Maintenance

• Tasks Automated at 11g

Optimizer Statistics Collection – statistics collection Segment Advisor – segment reorganization SQL Tuning Advisor – attempts to tune high-load SQL

Page 48: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Managing Performance in 11g : Statistics

• Oracle automatic statistics gathering 10g vs. 11g • Utilize optimizer_dynamic_sampling (KB32005)

http://resources.arcgis.com/content/kbase?fa=articleShow&d=32005

• Other options - ArcCatalog and GP Tools - sdetable –o update_dbms_stats –t roads –m estimate

• Temporary Tablespace - increase size for compute - consider increasing the size of your temporary table space to compute

statistics rather than estimate them because it provides more accurate statistics for the optimizer.

Page 49: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

11g Auto Statistics Gathering

• Setting Stale Percentage

Page 50: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Other Configuration Tips

• D-Table indexes - need to be checked for errors and redundancy in the _IDX

(KB31081) and _PK (KB24925) indexes • A-Table Index

- Performance gains by altering the Adds table indexes (KB32184) • Cache the lobs

- Most lobs can be cached to reduce trips to disk ST(KB33428), Network(34485), Raster(35521)

• Index rebuild - Sweeper Indexes need to be rebuilt to reduce their number of

blocks (KJ35409) • ST_GEOMETRY Spatial index shrink

- Reduce the number of blocks used by the spatial index (KB33341)

• Re-gather Stats - Do not build stats on the logfile tables

Note: Scripts to perform these tasks are included at the end of slide deck

Page 51: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Diagnostic Files

• Log files are ArcSDE output about what SDE is doing

• Useful when encountering a problem such as

- Service will not start, log file will tell you why

• ArcSDE logs stored in SDEHOME\etc - giomgr_<service>.log – Client connection info - sde_<service>.log – Application Server error messages - sdedc_Oracle.log – Direct Connect error messages

• Oracle log files, contain DBMS logged information - System alert log: alert_<sid>.log - Background process logs

Page 52: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle Trace Output – using tkprof The output contains SQL execution paths, row counts and wait events

OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows

------- ------ -------- ---------- -------- ---------- ---------- -------

Parse 0 0.00 0.00 0 0 0 0

Execute 16 0.17 0.19 0 0 0 0

Fetch 189 2.96 3.53 0 1605 0 17820

------- ------ -------- ---------- ---------- ---------- --------- --------

205 3.14 3.72 0 1605 0 17820

Misses in library cache during parse: 0

Elapsed times include waiting on following events:

Event waited on Times Max. Wait Total Waited

---------------------------------------- Waited ---------- ------------

SQL*Net message to client 200 0.00 0.00

SQL*Net message from client 199 0.52 2.50

SQL*Net more data to client 923 0.00 0.09

direct path read 44 0.05 0.15

log file sync 12 0.00 0.00

Page 53: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Capturing Metrics: Session Information

KB30394 – HowTo: Enable extended Oracle tracing for an ArcGIS connection http://resources.arcgis.com/content/kbase?fa=articleShow&d=30394

Opened cursors cumulative 3468 Opened cursors current 34

• Check Performance Metrics

• Review Statistics

• Search Sessions

• Select the Correct Session sde.process_information v$session, v$process

Page 54: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Measuring and Monitoring Memory

• Tools available in 11g – Manager Advisor Central - Memory Advisors

Page 55: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

SDE Logfiles

• Three types - giomgr_<service>.log

- giomgr process information, startup/shutdown, connections

- sde_<service>.log - gsrvr process information, event, errors - Setting %SDEVERBOSE% to true increase the amount of

information logged, but slows performance

- sde_setup.log - Information about the setup process

• To enable for direct connections, set %SDEHOME%, or check the user %TEMP% directory

Page 56: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

SDE DC Log

• Information that is similar to the sde_<service>.log file.

- This file is only present during direct connections

• This file will automatically be generated and placed in the %SDEHOME%/etc or %TEMP% directories.

Page 57: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

SDE Trace

- This logfile lists all the calls that are made to the ArcSDE client API.

- Not all of these calls result in a round trip to the server - To enable set %SDETRACELOC% to the path name.

Files are named %SDETRACELOC%.001, %SDETRACELOC%.002…

Page 58: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

SDE Intercept

• A list of all calls between the SDE client and the SDE sever.

- If using DC, will have two files that mirror each other.

• To enable set: - %SDEINTERCEPT% to the required values

- set SDEINTERCEPT=cwrfT - %SDEINTERCEPTLOC% to the path name

- Files are named %SDEINTERCEPTLOC%.001, %SDEINTERCEPTLOC%.002…

Page 59: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle DBMS Log

• Oracle Alert Log, - “The Alert file is a log file that records information about

internal errors and administrative activities, such as backups.” http://download.oracle.com/docs/cd/E11882_01/server.112/e17766/intro.htm#i10817

• Automatically enabled, location can be found by issuing the following SQL

- SQL> select value from v$parameter where name = 'background_dump_dest'

Page 60: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle SQL Trace

• Description of every SQL statement that was executed per session

- Very useful in analysis of slow running queries and operations, TKProf results much more readable

• Activate by altering session - As DBA user, tracing a different session:

- SQL> DBMS_SYSYEM.SET_ENV(<session id>, <session serial>, 10046, 12, ‘’)

- Tracing the current session, additional privileges required: - SQL> alter session set events '10046 trace name context forever, level 12'

Page 61: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Summary

Log Type 2 Tier 3 Tier Location SDE Logfiles set SDEHOME Active by Default %SDEHOME%/etc

/

SDE Trace set SDETRACELOC

Set SDETRACELOC

%SDETRACELOC%

SDE Intercept set SDEINTERCEPT and set SDEINTERCEPTLOC

set SDEINTERCEPT and set SDEINTERCEPTLOC

%SDEINTERCEPTLOC%

SDE DC Log Not Available Active %TEMP%

Oracle DBMS Log

Active Active Oracle Alert Log

Oracle SQL Trace

‘DBMS_SYSYEM.SET_ENV…’ or ‘alter session set events …’

‘DBMS_SYSYEM.SET_ENV…’ or ‘alter session set events …’

Oracle User Dump Dest

Page 62: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Spatial Clustering and Reorg

Performance and Troubleshooting

Demo

Page 63: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Advanced Configuration Topics

Page 64: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Advanced Configuration Topics

• Some not formally supported – must test • External Authentication – Direct Connect

- OS Authentication - Advanced Security - Encryption - Must use Direct Connect

• Oracle RAC – clusters - Must use Direct Connect

• Oracle DataGuard – replication - Physical – “stand by” - Logical

Page 65: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle RAC

• Use Direct Connect architecture • Extproc configuration for ST_Geometry spatial type

- Windows ST_ShapeLib.dll is dependent on dlls found in the Visual C++ Redistribution Package.

- Installed by SDE setup program automatically, or can be installed using redistribution package

• Cursors - Cursors don’t failover, reconnect either must be done or takes

place depending upon application and configuration.

Page 66: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Oracle Dataguard

• Physical – standby database – commonly used • Logical – RDBMS replication - rare in Geodatabase

configurations - Users can connect to “standby” database, but should be

treated as “read-only”. - Not supported, and configuration complex due to SQL

exclusion required.

Page 67: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

• Spatial Functions can be used in Spatial Views - Can provide “dynamic” feature classes on map

• Also Materialized Views • May require registration with ArcSDE and

Geodatabase - Depending upon creation method used

- sdetable –o create_view vs. SQL - Depending upon intended use

- Display in arcmap or for sql query only, Query Layer (10.x)

Views, Spatial Views and Types

CREATE VIEW san_berdoo_quakes_v AS SELECT a.objectid, a.location, b.name FROM quakes4 a, st_counties b WHERE b.name = 'San Bernardino' AND sde.st_intersects(a.location,b.boundary);

Page 68: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Advanced View Customization

Page 69: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Advanced View Customization

select /* FIRST_ROWS */ service_point_id, shape, ckt_id, job_id, poly_id, prev_energ_poly_id, lock_status, xfr_tag, point_type, account_priority, customer_count, replace_flag, clue_cd, pri_call_type, priority, topcall, symbol From (select g.*, gc.clue_cd, m.pri_call_type, ROW_NUMBER() OVER (PARTITION BY g.service_point_id ORDER BY m.priority) topcall, m.priority, m.symbol from gator_service_points g, gator_calls gc, master_lut m Where g.service_point_id = gc.service_pt_id And CASE when g.job_id is null then 0 else 1 end = m.assign And gc.clue_cd = m.clue and g.account_priority = m.accnt and (CASE When ROUND((sysdate - gc.last_chng_datetime) * 1440) = 0 then '0' When ROUND((sysdate - gc.last_chng_datetime) * 1440) > 0 and ROUND((sysdate - gc.last_chng_datetime) * 1440) <= 120 then '120' When ROUND((sysdate - gc.last_chng_datetime) * 1440) > 120 and ROUND((sysdate - gc.last_chng_datetime) * 1440) <= 240 then '240' When ROUND((sysdate - gc.last_chng_datetime) * 1440) > 240 then '241' Else '9999‘ End) = m.age_class) where topcall = 1 and pri_call_type IN ('PF','LD','LO')

Page 70: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Spatial Functions Example

select c.objectid, sde.st_intersection(c.shape, b.shape) shape from contour_l c, brokentopv b where sde.st_intersects(c.shape, b.shape)=1

Page 71: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Bulk Loading Considerations

• Exception to the rule Versioning

- Do a reconcile and post of the version to default before you load - Highly recommended that you do a compress before the load - No other editing during a load

- Indexes - Bulk loads are faster without indexes - Put feature classes in load only mode (drop indexes) before

loading - Do not forget to take them out again of load only mode (recreate

indexes)

Page 72: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Why to export/import using RDBMS tools?

Target has SDE – without GDB functionality

Target Has SDE – with GDB functionality

Target Doesn’t have SDE

Source Has SDE

Use database tools

Use GDB replication

Use database tools

• Copying non-versioned data without GDB functionality

• Copying the complete database to a test instance

• Logical Rules:

Page 73: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

What to Export?

• No GDB functionality : SDE.ST_GEOMETRY and SDO_GEOMETRY data, or tables without spatial data, with no GDB functionality (Domains, Relationship Classes, Feature Dataset)

- Only need to export the table - Drop SDE.ST_Geometry spatial prior to export

• GDB functionality need the SDE schema. - The User, all related data and SDE schema - May be easier to export entire Oracle Instance

Page 74: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

How to export?

• Data pump (Oracle), SDEExport (ArcSDE) - Oracle tools, work well with ST_Geometry starting at 9.3

• Simple data only. Advanced object types have to be exported using the ArcGIS, GP tools, python or ArcObjects.

• KB34342 Index: Working with ST_Geometry and Oracle export/import

Page 75: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Backup Options

• Backup all schemas, including SDE • Make sure you test your recovery scripts/backups

before you need to use them.

• On restore, may have to complie SDE schema packages.

- DBMS_UTILLiTY.COMPILE_SCHEMA(‘SDE’)

Note: It is worthwhile to backup dbinit file and dbtune settings whenever they change.

Page 76: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Steps to evaluate UC sessions

• My UC Homepage > “Evaluate Sessions”

• Choose session from planner OR

• Search for session

www.esri.com/ucsurveysessions

Page 77: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

• Thank you for attending

• Have fun at UC2012

• Open for Questions

• email:

[email protected] [email protected]

• Please fill out the evaluation:

www.esri.com/ucsessionsurveys

Page 78: Administering your Oracle Geodatabase · Oracle Memory: -System Global Area (SGA) and Program Global Area (PGA) -Memory areas used for - Interpreting SQL statements, Fetching data

Display Index Grid Stats


Recommended