+ All Categories
Home > Documents > Oracle Autonomous Data Warehouse: These Are the...

Oracle Autonomous Data Warehouse: These Are the...

Date post: 07-Jul-2020
Category:
Upload: others
View: 2 times
Download: 0 times
Share this document with a friend
31
Oracle Autonomous Data Warehouse: These Are the Droids You’re Looking For November 1, 2018 Jim Czuprynski President Zero Defect Computing, Inc. @ JimTheWhyGuy
Transcript
Page 1: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

Oracle Autonomous Data Warehouse:These Are the Droids You’re Looking For

November 1, 2018

Jim Czuprynski

President

Zero Defect Computing, Inc.

@JimTheWhyGuy

Page 2: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Jim Czuprynski: Credentials

• 35+ years of database-centric IT experience

• Oracle DBA since 2001

• Oracle 9i, 10g, 11g, 12c OCP

• Oracle ACE Director

• 100+ articles on databasejournal.com and IOUG SELECT

• Co-author of 4 Oracle books

• Oracle-centric blog (Generally, It Depends)

• Regular speaker at Oracle OpenWorld, IOUG COLLABORATE, Hotsos Symposium, and Regional OUGs

E-mail me at [email protected] me on Twitter (@JimTheWhyGuy)Connect with me on LinkedIn (Jim Czuprynski)

Page 3: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Our Agenda

•Autonomous Data Warehouse (ADW): The Basics

•Creating, Controlling, and Monitoring an ADW Instance

•Monitoring ADW Performance in Multiple Dimensions

•What Happens If My ADW Instance Becomes [Over]whelmed?

• Loading and Migrating Data Into ADW

•Conclusions, Use Cases, and What It Means For Oracle DBAs

Page 4: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

3 Membership Tiers• Oracle ACE Director• Oracle ACE• Oracle ACE Associate

bit.ly/OracleACEProgram

500+ Technical Experts Helping Peers Globally

Connect:

Nominate yourself or someone you know: acenomination.oracle.com

@oracleace

Facebook.com/oracleaces

[email protected]

Page 5: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

COLLABORATE is the largest user-led, user-focused event for the Oracle community. Featuring 1,000+ education and networking opportunities. Discover solutions for

wherever you are in your technology journey.

COLLABORATE 19April 7-11, 2019

Gonzalez Convention Center San Antonio, Texas

Registration is now open!http://collaborate.ioug.org/register

Page 6: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

Call for Presentations Now Open

Submission Tracks• Data Science & Analytics• Database• DevOps and Deploy• Personal Growth & Leadership

Submit your abstract by November 2 for the opportunity to share your expertise and receive complimentary registration to COLLABORATE 19.

collaborate.ioug.org/c19callforspeakers

Page 7: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Is Autonomosity Threatening Your Job? Well … You Are Not Alone.

If you are nervous about whether Autonomous Database is a threat to

your Oracle DBA job …

… then you are not the only oneworried about automation’s impact.

Page 8: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Moving to ADW: A Suggested Business Process Flow

Assess

• Is my application

workload really ready to move

to ADW?

Plan

• What migration strategy is most

appropriate?

• How long of an outage can my

production application

afford?

Migrate

• Transfer data using chosen

migration strategy, and

keep it synchronized

Monitor

• Watch for any unexpected

service outage / performance degradation /

user complaints

Tweak

• Should any application

workloads shift to a different ADW instance service?

As an evolving Oracle Data Enterprise Architect, it’s crucial to recognize and embrace the main thrust of ADW:

No More Knobs!

Page 9: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Creating a New Instance (1)

Specify your cloud account …1

… and get logged in2

Access your Cloud Dashboard …3

… and choose what kind of instance to create4

Build a new compartment for your ADWC instance …5

… and check out the other compartments available6

Page 10: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Creating a New Instance (2)

Specify a compartment and administrator credentials …

1 … and ADW instance creation begins!2

ADW instance now shows up in chosen compartment …3

… and your first ADW instance is now ready to access4

Page 11: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Creating a New Instance (3)

Connect to the new instance using the ADMIN account …

1Here’s your first look at the ADW Service Console!2

Request new credentials for access …3

… supply a robust password …4

… and save the new credentials in TNSNAMES home

5

Page 12: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Some Unexpected Surprises! (Well, At Least At First.)

SQL> CREATE INDEX tpcds.store_sales_cdemo_idx

ON tpcds.store_sales (ss_cdemo_sk);

2 ON tpcds.store_sales (ss_cdemo_sk)

*

ERROR at line 2:

ORA-01031: insufficient privileges

Creating a non-unique, “unconstrained” index:1

Hmmm … that’s interesting!

SQL> CREATE TABLE tpcds.customer_credit_ratings (

ccr_customer_number NUMBER(7)

,ccr_last_reported DATE

,ccr_credit_rating NUMBER(5)

,ccr_missed_payments NUMBER(3)

,ccr_credit_maximum NUMBER(7)

)

STORAGE (INITIAL 8M NEXT 4M)

PARTITION BY RANGE (ccr_last_reported)

INTERVAL(NUMTOYMINTERVAL(3, 'MONTH'))

(PARTITION ccr_oldest

VALUES LESS THAN (TO_DATE('1998-04-01', 'yyyy-mm-dd’)));

SQL> ,ccr_last_reported DATE

*

ERROR at line 30:

ORA-00439: feature not enabled: Partitioning

Building a partitioned

table:

2

Wait … what? But this is a data

warehouse!

Page 13: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Some Unexpected Surprises - Satisfied! (2)

SQL> CREATE INDEX tpcds.store_sales_cdemo_idx

ON tpcds.store_sales (ss_cdemo_sk);

Index TPCDS.STORE_SALES_CDEMO_IDX created.

Creating a non-unique, “unconstrained” index:1

Good! Secondaryindexes are now

permitted.

SQL> CREATE TABLE tpcds.customer_credit_ratings (

ccr_customer_number NUMBER(7)

,ccr_last_reported DATE

,ccr_credit_rating NUMBER(5)

,ccr_missed_payments NUMBER(3)

,ccr_credit_maximum NUMBER(7)

)

STORAGE (INITIAL 8M NEXT 4M)

PARTITION BY RANGE (ccr_last_reported)

INTERVAL(NUMTOYMINTERVAL(3, 'MONTH'))

(PARTITION ccr_oldest

VALUES LESS THAN (TO_DATE('1998-04-01', 'yyyy-mm-dd’)));

Table TPCDS.CUSTOMER_CREDIT_RATINGS created.

Building a partitioned

table:

2

Aha! So thisworks now! After all, this is a data

warehouse!

Page 14: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Monitoring Instance and Statement Performance

How is the ADW instance performing right now, and are there any evident “pushbacks” against a running workload?1 Performance can also be viewed for a particular narrower time period2

Viewing the performance of running as well as completed individual statements3Viewing an individual SQL statement’s performance …4 … and the statement’s execution plan5

Terminating a “wayward” session:6

Page 15: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Automatic Monitoring of SQL Statement Performance

SQL statements are automatically monitored for performance when they exceed 5 seconds of CPU or I/O activity …

1 … and the actual operations consuming the most resources are easy to identify while the statement’s still running

2Individual aspects of SQL statement activity are easy to review …3 … as well as specific CPU, memory, and I/O metrics4

Page 16: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Automatically Provided Database Services

Once an 18c ADW database is created, three database services are also automatically provisioned:

• Format: [adw instance name]_{HIGH | MEDIUM | LOW}

• Each service provides different levels of application resource provisioning

• Leverages Database Resource Manager (DBRM) and I/O Resource Manager (IORM)

Service NameResources

Per Statement

Concurrent SQL

StatementsImplications of These Limitations:

TPCDS_HIGH High 3 Any one statement can use all CPUs and all I/O resources

TPCDS_MEDIUM MediumMultiple of

OCPUs (linear scaling)

Any one statement can use all CPUs and multiple I/O resources

TPCDS_LOW Low 2X OCPUs Any one statement can use a single CPU and multiple I/O resources

See the detailed documentation for complete information on how these database services work.

Page 17: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: But What If I Run Out of Resources?

Workload Exhaustion Demonstration using TPC-DS Top 15 “Ugly” Queries

Workload Distribution:3 – HIGH

6 – MEDIUM8 - LOW

Start-out: 4 OCPUs …

Ramp-up:6 OCPUs …

Ramp-down: 2 OCPUs …

Page 18: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Database Feature Limitations

Several database features normally available for an OCI-

resident Oracle database are restricted for ADW instances:

Object / Permission / Feature Restrictions

Tablespaces Cannot be added, removed, or modified

Parallelism Enabled by default, based on number of OCPUs and chosen database service for application to connect

Compression HCC compression is enabled by default; compression method is changeable

Result Caching Enabled by default for all statements; cannot be changed

Node File System and OS No direct access permitted

Database Links to Other Databases Prohibited to preserve security features

PL/SQL Calls Using DB Links Likewise, prohibited

Parallel DML Enabled by default, but can be disabled at session level:

ALTER SESSION DISABLE PARALLEL DML;

See Restrictions for Database Features for a complete list of these ADW limitations.

Page 19: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Unavailable SQL Commands

The following SQL commands cannot be executed against an ADW instance:

See Restrictions for SQL Commands for complete information on these unavailable SQL commands.

SQL Command Reason for Unavailability

ADMINISTER KEY MANAGEMENT

PDB-level security tightly enforced

CREATE / ALTER / DROP TABLESPACE

Tablespaces are strictly controlled

ALTER PROFILE Resource limits and security restraints tightly enforced

CREATE DATABASE LINK Self-containment and security

Page 20: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Permitted Changes to Initialization Parameters

Only the following database initialization parameters may be modified:

Initialization Parameters That Can Be Modified

APPROX_FOR_AGGREGATION OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES*

APPROX_FOR_COUNT_DISTINCT OPTIMIZER_IGNORE_HINTS

APPROX_FOR_PERCENTILE OPTIMIZER_IGNORE_PARALLEL_HINTS

AWR_PDB_AUTOFLUSH_ENABLED PLSCOPE_SETTINGS

PLSQL_DEBUG

PLSQL_WARNINGS

PLSQL_OPTIMIZE_LEVEL

PLSQL_CCFLAGS

Most NLS parameters TIME_ZONE*

* Only via ALTER SESSION

See Restrictions to Database Initialization Parameters for more information on permissible changes.

Page 21: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Unavailable Options and Packs

The following database options and packs are not enabled for ADW instances:

Non-Usable Database Options and Packs

Oracle Application Express Oracle Spatial and Graph

Oracle Tuning Pack Oracle Data Masking and Subsetting Pack

Oracle Real Application Testing Oracle R capabilities of Oracle Advanced Analytics

Oracle Database Vault Oracle Industry Data Models

Oracle Data Masking and Subsetting Pack Oracle Text

Oracle Database Lifecycle Management Pack Oracle Multimedia

Oracle Cloud Management Pack for Oracle

Database

Java in DB

Oracle OLAP Oracle XML DB

Oracle Workspace Manager Context

See Restrictions for Database Features for complete information on unusable database options and packs.

Page 22: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Manual vs. Autonomous Database: Comparison of Capabilities

22

Am I Empowered To … 18c ADW

Add my own schemas? Yes Yes

Connect applications directly via TNSNAMES? Yes Yes

Elastically upsize or downsize CPUs, memory, and storage? Yes Yes

Create my own CDBs and PDBs? Yes No

Clone a PDB to the same or another CDB? Yes No

Build my own tablespaces? Yes No

Modify memory pool sizes (e.g. SGA_SIZE)? Yes No

Modify security settings (e.g. keystores)? Yes No

Connect directly as SYS? Yes No

Build a PDB using RMAN backups? Yes No

Connect with Enterprise Manager Cloud Control for monitoring?Via Proxy

AgentNo

Page 23: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Migrating and Loading Data

23

Am I Empowered To … 18c ADW

Load data with SQL*Loader or SQLDeveloper? YesYes, but source files

reside in object storage

Load data with Data Pump Import? YesYes, but export dump set resides in object storage

Export data with Data Pump Export? YesYes, but export dump set resides in object storage

Synchronize data with GoldenGate*? Yes Yes, within certain limits

*See this documentation for complete information on GoldenGate capabilities

for Autonomous Databases.

Page 24: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Loading Data Via DBMS_CLOUD.COPY_DATA (1)

Creating credentials for accessing file system:1

SQL> BEGIN

DBMS_CLOUD.CREATE_CREDENTIAL(

credential_name => 'extb_tpcds'

,username => '[email protected]'

,password => '(;n<T1#-MpY>4u>_yilK'

);

END;

/

Creating the new table:2

SQL> CREATE TABLE tpcds.customer_credit_ratings (

ccr_customer_number NUMBER(7)

,ccr_last_reported DATE

,ccr_credit_rating NUMBER(5)

,ccr_missed_payments NUMBER(3)

,ccr_credit_maximum NUMBER(7)

)

STORAGE (INITIAL 8M NEXT 4M)

PARTITION BY RANGE (ccr_last_reported)

INTERVAL(NUMTOYMINTERVAL(3, 'MONTH'))

(PARTITION ccr_oldest

VALUES LESS THAN (TO_DATE('1998-04-01', 'yyyy-mm-dd'))

);

Table created.

Loading data with DBMS_CLOUD.COPY_DATA:3

SQL> BEGIN

DBMS_CLOUD.COPY_DATA(

table_name => 'CUSTOMER_CREDIT_RATINGS'

,credential_name => 'EXTB_TPCDS'

,file_uri_list =>

'https://swiftobjectstorage.us-ashburn-1.oraclecloud.com/v1/iougcloudtrial/

ADWExternalTables/CreditScoring_Current.dat'

,schema_name => 'TPCDS'

,field_list => 'ccr_customer_number CHAR(08),ccr_last_reported CHAR(10)

,ccr_credit_rating CHAR(05),ccr_missed_payments CHAR(03)

,ccr_credit_maximum CHAR(07)’

,format => '{"delimiter" : "|" , "dateformat" : "YYYY-MM-DD"}');

EXCEPTION

WHEN OTHERS THEN

DBMS_OUTPUT.PUT_LINE('ERROR:' || SQLCODE || ' ' || SQLERRM);

END;

/

PL/SQL procedure successfully completed.

Page 25: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Loading Data Via DBMS_CLOUD. COPY_DATA (2)

Monitoring a running load task …1 … even when it fails to complete successfully! 2Show status of running load operations:

3SET LINESIZE 132

SET PAGESIZE 20000

COL owner_name FORMAT A08 HEADING "Owner"

COL table_name FORMAT A24 HEADING "Table|Loaded"

COL type FORMAT A08 HEADING "Operation"

COL status FORMAT A10 HEADING "Status"

COL start_dtm FORMAT A19 HEADING "Started At"

COL update_dtm FORMAT A19 HEADING "Finished At"

COL logfile_table FORMAT A12 HEADING "LOGFILE|Table"

COL badfile_table FORMAT A12 HEADING "BADFILE|Table"

SELECT

owner_name

,table_name

,type

,status

,TO_CHAR(start_time,'YYYY-MM-DD HH24:MI:SS') start_dtm

,TO_CHAR(update_time,'YYYY-MM-DD HH24:MI:SS') update_dtm

,logfile_table

,badfile_table

FROM user_load_operations

WHERE type = 'COPY'

ORDER BY start_time DESC;

Table LOGFILE BADFILE

Owner Loaded Operatio Status Started At Finished At Table Table

-------- ------------------------ -------- ---------- ------------------- ------------------- ------------ ------------

TPCDS CUSTOMER_CREDIT_RATINGS COPY COMPLETED 2018-10-08 11:00:59 2018-10-08 11:03:12 COPY$38_LOG COPY$38_BAD

TPCDS CUSTOMER_CREDIT_RATINGS COPY FAILED 2018-10-08 10:51:09 2018-10-08 10:53:16 COPY$37_LOG COPY$37_BAD

TPCDS CUSTOMER_CREDIT_RATINGS COPY FAILED 2018-10-08 10:50:49 2018-10-08 10:50:49

TPCDS CUSTOMER_CREDIT_RATINGS COPY FAILED 2018-10-08 10:50:03 2018-10-08 10:50:03

TPCDS CUSTOMER_CREDIT_RATINGS COPY FAILED 2018-10-08 10:34:33 2018-10-08 10:35:56 COPY$34_LOG COPY$34_BAD

Show the resultingLOG File:

4

SQL> SELECT *

FROM copy$38_log;

LOG file opened at 10/08/18 16:03:01

Total Number of Files=1

Data File: https://swiftobjectstorage.us-ashburn-1.oraclecloud.com/v1/iougcloudt

Log File: COPY$38_144722.log

LOG file opened at 10/08/18 16:03:01

Bad File: COPY$38_355882.bad

Field Definitions for table COPY$WQPDD1Q3X2892USR6RY7

Record format DELIMITED BY

Data in file has same endianness as the platform

Rows with all null fields are accepted

Fields in Data Source:

CCR_CUSTOMER_NUMBER CHAR (8)

Terminated by "|"

CCR_LAST_REPORTED CHAR (10)

Date datatype DATE, date mask YYYY-MM-DD

Terminated by "|"

CCR_CREDIT_RATING CHAR (5)

Terminated by "|"

CCR_MISSED_PAYMENTS CHAR (3)

Terminated by "|"

CCR_CREDIT_MAXIMUM CHAR (7)

Terminated by "|"

Date Cache Statistics for table COPY$WQPDD1Q3X2892USR6RY7

Date conversion cache disabled due to overflow (default size: 1000)

Page 26: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Migrating Data Via DataPump Export and Import

Export data from source database:1

$> expdp vevo/vevo@pdbvevo parfile=ADW_VEVO.expdp

#####

# File: ADW_VEVO.expdp

# Purpose: DataPump Export parameter file for VEVO schema

# 1.) Exclude all:

# - Clusters

# - Database Links

# - Indexes and Index Types

# - Materialized Views, Logs, and Zone Maps

# 2.) For partitioned tables, unload all table data in a single operation (rather

# than unload each table partition as a separate operation) for faster loading

# 3.) Use 4 degrees of parallelism and write to multiple dump files

#####

DIRECTORY=DATA_PUMP_DIR

EXCLUDE=INDEX, CLUSTER, INDEXTYPE, MATERIALIZED_VIEW, MATERIALIZED_VIEW_LOG,

MATERIALIZED_ZONEMAP, DB_LINK

DATA_OPTIONS=GROUP_PARTITION_TABLE_DATA

PARALLEL=4

SCHEMAS=vevo

DUMPFILE=export%u.dmp

Export: Release 18.0.0.0.0 - Production on Sat Sep 1 19:12:37 2018

Version 18.1.0.0.0

Copyright (c) 1982, 2018, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 18c EE Extreme Perf Release 18.0.0.0.0 - Production

Starting "VEVO"."SYS_EXPORT_SCHEMA_01": vevo/********@pdbvevo parfile=ADW_VEVO.expdp

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY

Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA

Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA

. . .

Processing object type SCHEMA_EXPORT/TABLE/TABLE

Processing object type SCHEMA_EXPORT/STATISTICS/MARKER

. . exported "VEVO"."T_CANVASSING" 27.77 MB 539821 rows

. . exported "VEVO"."T_STAFF" 13.86 KB 26 rows

. . exported "VEVO"."T_CAMPAIGN_ORG" 8.031 KB 25 rows

. . exported "VEVO"."T_VOTING_RESULTS" 80.90 MB 1887761 rows

. . exported "VEVO"."T_VOTERS" 84.84 MB 180000 rows

ORA-39173: Encrypted data has been stored unencrypted in dump file set.

Master table "VEVO"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded

******************************************************************************

Dump file set for VEVO.SYS_EXPORT_SCHEMA_01 is:

/u01/app/oracle/admin/ORCL/dpdump/6F4634904CBF29F3E0535AEA110A9CAE/export01.dmp

/u01/app/oracle/admin/ORCL/dpdump/6F4634904CBF29F3E0535AEA110A9CAE/export02.dmp

/u01/app/oracle/admin/ORCL/dpdump/6F4634904CBF29F3E0535AEA110A9CAE/export03.dmp

/u01/app/oracle/admin/ORCL/dpdump/6F4634904CBF29F3E0535AEA110A9CAE/export04.dmp

Job "VEVO"."SYS_EXPORT_SCHEMA_01" successfully completed at Sat Sep 1 19:13:17 2018 elapsed 0 00:00:39

Transfer export dump set to Object Container2

Set up credentials for access:3

SQL> CONNECT admin/'N0M0reKn0bs#@tpcds_high;

SQL> BEGIN

DBMS_CLOUD.CREATE_CREDENTIAL(

credential_name => ‘DPI_VEVO’

,username => '[email protected]'

,password => 'N0M0reKn0bs#'

);

END;

/

SQL> ALTER DATABASE PROPERTY

SET default_credential = 'ADMIN.DPI_VEVO';

Add new schema into ADW Instance:4

SQL> CONNECT admin/'N0M0reKn0bs#@tpcds_high;

CREATE USER vevo

IDENTIFIED BY N0M0reKn0bs#

TEMPORARY TABLESPACE temp

PROFILE DEFAULT;

GRANT RESOURCE TO vevo;

GRANT CREATE PROCEDURE TO vevo;

GRANT CREATE PUBLIC SYNONYM TO vevo;

GRANT CREATE SEQUENCE TO vevo;

GRANT CREATE SESSION TO vevo;

GRANT CREATE SYNONYM TO vevo;

GRANT CREATE TABLE TO vevo;

GRANT CREATE VIEW TO vevo;

GRANT DROP PUBLIC SYNONYM TO vevo;

GRANT EXECUTE ANY PROCEDURE TO vevo;

GRANT READ,WRITE ON DIRECTORY data_pump_dir TO vevo;

Import data into ADW instance:5

$> ./impdp admin/IOUG1sAwesome@TPCDS_HIGH \

DIRECTORY=DATA_PUMP_DIR \

VERSION=18.0.0 \

REMAP_SCHEMA=vevo:vevo \

DUMPFILE=default_credential:https://swiftobjectstorage.us-ashburn-

1.oraclecloud.com/v1/iougcloudtrial/DP_VEVO/export%U.dmp \

PARALLEL=4 \

PARTITION_OPTIONS=MERGE \

TRANSFORM=SEGMENT_ATTRIBUTES:N \

TRANSFORM=DWCS_CVT_IOTS:Y \

TRANSFORM=CONSTRAINT_USE_DEFAULT_INDEX:Y \

EXCLUDE=index,cluster,indextype,materialized_view,materialized_view_log

,materialized_zonemap,db_link

Import: Release 12.2.0.1.0 - Production on Sun Sep 2 21:51:32 2018

Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 18c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

Master table "ADMIN"."SYS_IMPORT_FULL_02" successfully loaded/unloaded

Starting "ADMIN"."SYS_IMPORT_FULL_02": admin/********@TPCDS_HIGH DIRECTORY=DATA_PUMP_DIR VERSION=18.0.0

. . .

Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA

Processing object type SCHEMA_EXPORT/TABLE/TABLE

Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA

. . imported "VEVO"."T_STAFF" 13.86 KB 26 rows

. . imported "VEVO"."T_CANVASSING" 27.77 MB 539821 rows

. . imported "VEVO"."T_CAMPAIGN_ORG" 8.031 KB 25 rows

. . imported "VEVO"."T_VOTING_RESULTS" 80.90 MB 1887761 rows

. . imported "VEVO"."T_VOTERS" 84.84 MB 180000 rows

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT

Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

Processing object type SCHEMA_EXPORT/STATISTICS/MARKER

Job "ADMIN"."SYS_IMPORT_FULL_02" successfully completed at Mon Sep 3 02:52:42 2018 elapsed 0 00:01:06

Page 27: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Advantages of “No More Knobs“

Remember, ADW is all about no more knobs … and that’s really advantageous!

• Service instance can be stopped and restarted as necessary• Useful for conserving Cloud credits

• Easy to connect to • Only a few entries in SQLNET.ORA file and TNSNAMES.ORA are required

• Regular RMAN backups taken automatically on regular nightly schedule

• No instance tuning required• Memory pool sizes are already locked in

• Parallelism is automatically derived depending on number of OCPUs and service name selected for connection

• Only appropriate licensed options are included• No worries about accidently incurring potential additional licensing fees

• Direct-path loads are fully supported

• DataPump Export and Import provides for rapid provisioning from existing databases

• Support for GoldenGate has been added as well

Page 28: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Under the Covers of ADW: Some FAQs

Wow, ADW seems to be a bit primitive right now! Is it going to get more advanced?

Several new features are in the works for 19c ADW, including a reporting API that will show what action has been taken by

workload optimization AI/ML.

Not really, at least not yet in 18c … but stay tuned!

Can I review the results of the AI and Machine Learning algorithms that ADW has applied to

my workloads and objects?

Ummm … no, not yet. Does ADW automatically partition tables and indexes?

Yes, absolutely! Database In-Memory, Smart Scan, storage indexes, and Smart

Flash Cache are definitely in use.

Is ADW really using all of Exadata’s key features for most efficient execution of queries?

Page 29: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

ADW: Summary of Appropriate Use Cases

ADW is most appropriate for the following application workload requirements and environments:

• Read-only or read-mostly application workloads focused on leveraging analytical processing• All features of Exadata storage cell software – Smart Scan, Smart Storage, storage indexes - brought to bear on processing workloads• Tables are also cached in columnar format automatically with Exadata Smart Flash Cache• All query results are automatically saved within SQL Query Result Caches, thus limiting future re-read of same data• Up to 128 OCPUs and 128 TB of storage can be can be requested per ADW instance (subject to availability within instance’s region)

• Ideally, data warehousing application workload(s) should already be well-tuned to avoid surprises

• Virtually no DBA resources required for database management• No instance tuning is necessary• Selection of appropriate database service for the workload is really the only choice required• Parallelism derived from database service selected and number of OCPUs available• Scale-up and scale-down requires just a single button push

• Database migration and transformation only limited by desired / appropriate transferal methods• Fresh load: DBMS_CLOUD.COPY_DATA, SQL*Loader, or INSERT INTO … SELECT FROM an EXTERNAL Table • Existing schema(s): DataPump Import• Tight synchronization required: GoldenGate• Extremely large data transfers possible via Oracle Cloud Infrastructure Data Transfer Appliance

Page 30: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Either Drive a School Bus, or Program Its Replacement. Your Choice.

… or become a Data Enterprise Architect, ahead of the curve, ready to take advantage of

Autonomous Database and the future it portends

It’s all up to you.

You can be a school bus driver kind of DBA – oblivious to the elimination of

your job in just a few years …

Page 31: Oracle Autonomous Data Warehouse: These Are the …coug.us/wp/wp-content/uploads/2018/11/2019_ADW.pdf•Oracle 9i, 10g, 11g, 12c OCP •Oracle ACE Director •100+ articles on databasejournal.com

April 2-6, 2017 in Las Vegas, NV USA #C17LV

Useful Resources and Documentation

• ADW Documentation: https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/index.html

• Oracle Autonomous and Secure Cloud Services Blog: https://blogs.oracle.com/autonomous-and-secure-cloud-services

• Oracle Cloud Infrastructure Data Transfer Appliance: https://blogs.oracle.com/cloud-infrastructure/introducing-oracle-cloud-infrastructure-data-transfer-appliance

• Using GoldenGate to Replicate Data to ADW Cloud: https://docs.oracle.com/goldengate/c1230/gg-winux/GGODB/replicating-data-oracle-autonomous-data-warehouse-cloud.htm#GGODB-GUID-660E754E-B9A6-48DD-AA66-0D6B66A022CD


Recommended