Post on 14-Mar-2020
transcript
OUAG
SANGAM
AIOUG
DEPLOY A DISASTER RECOVERY SOLUTION AND SETUP STRESS TESTING CLUSTER USING CASCADED STANDBY
SESSSION# 201572
PR E PAR E D B Y :
RANDHIR KUMAR
O R AC L E D BA , G A L LU P
2
Oracle DBA at Gallup, an American research-based, global performance-management consulting organization.
Over 5yrs Experience in Oracle products.
Managed Oracle RAC database on RHEL6 server, handful experience in MYSQL
Done R&D projects
in.linkedin.com/in/imrandhirkumar
Speaker Introduction
3
Introduction
Why Cascaded Standby?
Setup Cascaded Standby
Setup Snapshot standby
Setup Stress Cluster
Conversion
Questions
Agenda
RANDHIR KUMAR
PHYSICAL+SNAPSHOT SNAPSHOT PHYSICAL STANDBY
4
Oracle Data Guard : One of the most efficient DR solution using Physical Standby.
Snapshot Standby: A hidden feature of oracle data guard is actually a fully updatable database can be used for stress testing.
INTRODUCTION
PRIMARY
;
How to combine both technology?
RANDHIR KUMAR
5
Why Cascaded Standby?
Allows usage of one Staging Standby as Snapshot Standby Database while other as a DR site.
No extra Bandwidth required, No Production performance overhead.
Also Receives redo from production in the background and ships it to Cascaded Standby.
PRIMARY STANDBY CASCADED STANDBY
SNAPSHOT
RANDHIR KUMAR
6
http://krandhir.wordpress.com/create-a-physical-standby-with-data-guard-broker/
Data Guard Physical Standby Setup
IVRD IVRDSB
PRIMARY PHYSICAL
DATA GURAD BROKER
RANDHIR KUMAR
7
11g Enterprise Edition Database (11.2.0.3) s/w with PSU11
Restrictions with 11.2.0.3 : A Physical Standby Database is the only Standby Database type that can cascade
redo.
The Data Guard broker does not support Cascaded Destinations in 11g.
A Cascaded Standby can’t be created using Active Duplication with active standby as target (Bug 18970361).
Environments
IVRD IVRDDR IVRDSB
PRIMARY STAGING CASCADED
8
Data Guard Architecture
SGA
ORL
ORL
Primary Database
Cascaded Standby Database
Use
r C
om
mit
REDO BUFFER
LGWR
SRL
SRL
ARCH
Database datafile
APPLY
Staging Standby Database
ARCH
LNS RFS APPLY Database datafile
ARCH
RFS ARCH
Database datafile
SRL
SRL
ARCH
RANDHIR KUMAR
9
Second line of defense.
Maintained at geographically remote place to meet DR aspect.
Part of 11g Enterprise Edition License.
Doesn’t supported by Data Guard broker.
Receives archives from either primary or staging standby database
ARCH to ARCH shipping. ASYNC always.
Cascaded Standby Database Cascaded Standby Database
RANDHIR KUMAR
10
Take a backup of Init.ora file and stop archive deletion on staging standby.
Copy password files from primary to respective $OH on Stress Cluster
Add TNS on Primary and staging standby for cascaded standby.
Add static entry for new standby in listener.ora file
Set parametes on both primary and standing standby. Log_Archive_Config=‘dg_config=(IVRD,IVRDSB,IVRDDR)’
Log_Archive_Dest_3='service=IVRDDR VALID_FOR=(STANDBY_LOGFILE,ALL_ROLES) DB_UNIQUE_NAME=IVRDDR'
Log_Archive_Dest_state_3=DEFER
On STAGING DATABASE:
FAL_SERVER=‘IVRD’ and FAL_CLIENT=‘IVRDSB’
CASCADED STANDBY SETUP Cascaded Standby Setup
RANDHIR KUMAR
11
Using RMAN “DUPLICATE FROM ACTIVE DATABASE” to create the Standby Database
Replace IVRD with IVRDDR in init file backed from primary.
Change few required parameters as per cascaded standby. CONTROFILE
LOCAL LISTENER and REMOTE LISTERERS
log_archive_dest_1
LOG_ARCHIVE_DEST_2='SERVICE=IVRD ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=IVRD’
LOG_ARCHIVE_DEST_STATE_2=DEFER
Log_Archive_Config==‘dg_config=(IVRD,IVRDSB,IVRDDR)’
FAL_CLIENT=IVRDDR
FAL_SERVER=IVRDSB,IVRD
*.log_file_name_convert='+DISKGROUP1/IVRD/ONLINE','+DISKGROUP6/IVRDDR/ONLINE'
*.db_file_name_convert='+DISKGROUP1/IVRD','+DISKGROUP1/IVRDSB‘
Enable redo transport from staging standby to cascaded standby
Start recovery on cascaded standby.
Cascaded Standby Setup Cascaded Standby Setup
RANDHIR KUMAR
12
Read-Write Mode(fully updatable)
Internally creates a Guaranteed Restore Point(GRP)
Receives REDO from primary but doesn’t not apply, ships to cascaded standby if enabled.
Single DGMGRL command creates GRP, cancels recovery, converts into snapshot and open in read right mode.
Flashback needs to be enabled if converting through DGMGRL
Stop archive deletion on Staging standby.
Snapshot Standby Database
SQL>ALTER SYSTEM SET DB_RECOVERY_FILE_DEST_SIZE=100G;
SQL>ALTER SYSTEM SET DB_RECOVERY_FILE_DEST='+FLASHBACK';
SQL> alter database flashback on;
RANDHIR KUMAR
13
Convert the staging standby into snapshot mode.
New Incarnation on cascaded standby due to conversion=> Terminate the sync up with Primary due to incarnation mismatch.
Setup Snapshot Standby Database
PRIMARY STANDBY CASCADED STANDBY SNAPSHOT
INCRANTION NO 2
INCRANTION NO 2
INCRANTION NO 2
INCRANTION NO 3
INCRANTION NO 3
open resetlogs
RANDHIR KUMAR
14
Solution Stop MRP on Cascaded Standby
Do the conversion
Restore the old incarnation on cascaded standby
Setup Snapshot Standby Database
SQL> alter database recover managed standby database cancel;
DGMGRL> connect sys/pwd
DGMGRL> CONVERT DATABASE 'IVRDSB' TO SNAPSHOT STANDBY;
DGMGRL> SHOW CONFIGURATION;
RMAN> list incarnation;
List of Database Incarnations
DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------- ------- -------- ------------ ---------- --------- ----------
2 2 IVRD 2093128578 PARENT 2412534122835 22-JUN-14
3 3 IVRD 2093128578 CURRENT 2419817575199 16-AUG-14
RMAN> reset database to incarnation 2;
RANDHIR KUMAR
15
Alert log info
Sun Oct 19 09:58:54 2014
MRP0: Background Media Recovery process shutdown (IVRDSB1)
Created guaranteed restore point
SNAPSHOT_STANDBY_REQUIRED_10/19/2014 09:58:54
Sun Oct 19 09:58:57 2014
RESETLOGS after complete recovery through change
2430651513612
Resetting resetlogs activation ID 2093143682 (0x7cc2d682)
Standby became primary SCN: 2430651513610
Setting recovery target incarnation to 8
CONVERT TO SNAPSHOT STANDBY: Complete - Database mounted as
snapshot standby
Completed: alter database convert to snapshot standby
Sun Oct 19 09:59:03 2014
ALTER DATABASE OPEN
Data Guard Broker initializing...
Data Guard Broker initialization complete
This instance was first to open
Picked broadcast on commit scheme to generate SCNs
Sun Oct 19 09:59:03 2014
Assigning activation ID 2103267323 (0x7d5d4ffb)
Sun Oct 19 09:59:07 2014
Completed: ALTER DATABASE OPEN
Sun Oct 19 09:59:37 2014
RFS[10]: Selected log 11 for thread 1 sequence 685 dbid
2093128578 branch 850929569
sequence 685 for destination LOG_ARCHIVE_DEST_3
Sun Oct 19 09:59:37 2014
Sun Oct 19 09:58:27 2014 Completed: alter database recover managed standby database cancel Sun Oct 19 09:59:08 2014 RFS[1]: New Archival REDO Branch(resetlogs_id): 861357539 Prior: 850929569 RFS[1]: Archival Activation ID: 0x0 Current: 0x7cc2d682 RFS[1]: Effect of primary database OPEN RESETLOGS RFS[1]: Incarnation entry added for Branch(resetlogs_id): 861357539 (IVRDDR1) Sun Oct 19 09:59:08 2014 Setting recovery target incarnation to 8 Archived Log entry 428 added for thread 2 sequence 1 rlc 861357539 ID 0x0 dest 3: RFS[1]: Opened log for thread 1 sequence 1 dbid 2093128578 branch 861357539 Archived Log entry 429 added for thread 1 sequence 1 rlc 861357539 ID 0x7d5d4ffb dest 3: Sun Oct 19 10:00:23 2014 Setting recovery target incarnation to 2 Sun Oct 19 09:59:30 2014 alter database recover managed standby database using current logfile disconnect Sun Oct 19 09:59:37 2014 RFS[2]: Selected log 10 for thread 1 sequence 685 dbid 2093128578 branch 850929569 Sun Oct 19 10:00:47 2014 Media Recovery Log +DISKGROUP6/ivrddr/1_685_850929569.arc Media Recovery Log Sun Oct 19 11:01:56 201Archived Log entry 485 added for thread 2 sequence 585 rlc 850929569 ID 0x7cc2d682 dest 3:
IVRDSB IVRDDR
RANDHIR KUMAR
16
START MRP ON Cascaded standby
Verify the redo shipping to standby even the Standby Database is in open mode. Check the block# column if its changing
Setup Snapshot Standby Database
SQL> alter database recover managed standby database using current
logfile disconnect;
STAGING STANDBY(SNAPSHOT STANDBY) SQL>SELECT STATUS, SEQUENCE#, BLOCK# FROM V$MANAGED_STANDBY WHERE
CLIENT_PROCESS in ('LGWR', 'ARCH');
STATUS SEQUENCE# BLOCK#
------------ ----------------- ------------
IDLE 564 376
IDLE 650 521
CASCADED STANDBY SQL>SELECT STATUS, SEQUENCE#, BLOCK# FROM V$MANAGED_STANDBY WHERE
CLIENT_PROCESS in ( 'ARCH');
STATUS SEQUENCE# BLOCK#
------------ ---------- ----------
CLOSING 518 1
CLOSING 627 1
RANDHIR KUMAR
17
Check the recovery on Cascaded standby when Staging standby is in snapshot mode.
Setup Snapshot Standby Database
On IVRDDR
SQL>select to_char(current_scn)
from v$database;
TO_CHAR(CURRENT_SCN)
-------------------------------
2428647321575
ON IVRD
SQL>select
scn_to_timestamp(2428647321575)
from dual;
SCN_TO_TIMESTAMP(2428647321575)
-------------------------------
02-OCT-14 07.02.23.000000000 AM
Alert log IVRDDR Sun Oct 19 10:02:19 2014 RFS[2]: Selected log 10 for thread 1 sequence 690 dbid 2093128578 branch 850929569 Sun Oct 19 10:02:19 2014 Archived Log entry 442 added for thread 1 sequence 690 ID 0x7cc2d682 dest 1: Archived Log entry 443 added for thread 2 sequence 4 rlc 861357539 ID 0x7d5d4ffb dest 3: RFS[1]: Opened log for thread 2 sequence 575 dbid 2093128578 branch 850929569 Archived Log entry 444 added for thread 2 sequence 575 rlc 850929569 ID 0x7cc2d682 dest 3: Archived Log entry 447 added for thread 1 sequence 6 rlc 861357539 ID 0x7d5d4ffb dest 3: Sun Oct 19 11:06:30 2014 alter database recover managed standby database using current logfile disconnect Sun Oct 19 11:06:31 2014 MRP0 started with pid=42, OS id=27661 MRP0: Background Managed Standby Recovery process started (IVRDDR1) started logmerger process Sun Oct 19 11:06:37 2014 Managed Standby Recovery starting Real Time Apply Sun Oct 19 11:06:37 2014 Media Recovery Log +DISKGROUP6/ivrddr/1_690_850929569.arc Media Recovery Log +DISKGROUP6/ivrddr/2_574_850929569.arc Media Recovery Log +DISKGROUP6/ivrddr/2_575_850929569.arc
RANDHIR KUMAR
18
Create and start Cluster Services for Snapshot Standby Databases
Change Database Links on Snapshot Standby’s to Standby Service names.
Change password of Application and User accounts to be different from production.
Comment the code for SMTP Server to prevent any unwanted emails.
Job Classes will not run as Job services point to primary services. So modify the job class attribute with new services
Change the DATABASE_ROLE to SNAPSHOT STANDBY for the jobs that you want to run on Standby Stress Cluster
Disable Materialized views not needed or dependent upon anything outside of Stress Cluster
Add Snapshot Standby databases to OEM for management
Enable Scheduler Jobs
Setup Stress Cluster
RANDHIR KUMAR
19
Job Classes will not run as Job services will be different. Follow the below to enable jobs
Change the DATABASE_ROLE to SNAPSHOT STANDBY for the jobs that you want to run on Standby Stress Cluster.
Scheduler Jobs in Snapshot Standby Mode
Alter Job Class to attach to new Job Services BEGIN
DBMS_SCHEDULER.SET_ATTRIBUTE (
name =>'IVR_JOB_CLASS ',
attribute => 'SERVICE',
value => 'IVRSBJOB');
end;
BEGIN
DBMS_SCHEDULER.SET_ATTRIBUTE (
name => ‘’TEST_REMINDER_EMAIL',
attribute => 'database_role',
value =>'SNAPSHOT STANDBY');
END;
END;
RANDHIR KUMAR
20
Conversion(Back to Physical) Delete snapshot standby archivelogs
Convert into Physical Standby
DISABLE FLASHBACK on Staging Database
Enable archive deletion on Staging Database.
Select 'delete noprompt archivelog '||''''||name||''';' from
v\$archived_log a where registrar='ARCH' and STATUS = 'A' order by
first_time desc;
DGMGRL> connect sys/pwd
CONVERT DATABASE 'IVRDSB' TO PHYSICAL STANDBY;
DGMGRL> SHOW CONFIGURATION
Alter database flashback off;
RANDHIR KUMAR
21
DBMS Scheduler for SNAPSHOT STANDBY bug 13399711
Switchover Bug 11719886
Cascaded Standby can’t be created using Active Duplication with active standby as target
18970361
Bugs and Patches