CERN IT Department CH-1211 Genève 23 Switzerland t Streams new features in 11g Zbigniew...

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

Streams new features in 11g

Zbigniew Baranowski

CERN IT Department

CH-1211 Genève 23


it

InternetServices

Zbigniew Baranowski 2

Outline

• Combined capture and apply• Automatic split and merge procedures• Compare and converge• Streams performance advisor• Other features

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Combined capture and apply• Optimization of data flow management• Uses direct connections between capture

and apply process• It is applied automatically if required

conditions are satisfied

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Combined capture and apply• CC&A requirements

– The capture process's queue must have a single publisher, and it must be the capture process

– A propagation must be configured between the capture process's queue and the apply process's queue. There can be no intermediate queues

– Each apply process that applies changes from the same source database must use a different queue

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Combined capture and apply• Improved flow control (new capture process

behavior)– No subscribers => capturing paused– Apply or propagation disabled => target removed from

subscribers list => no LCRs accumulation in source queue– Target re-enabled => target added to the subscribers list

=> automatic re-capturing of missing changes– Controlling the oldest applied SCN of subscribers– Constant max queues capacity

• 10000 messages for source queue• 1000 messages for destination queue

– No spilling from queues

• Max throughput increased about x10

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Combined capture and apply• Point in time recovery automatic handling

– No additional steps are required to restart replication

– Capture needs an access to the old archive logs before recovery

– Recovery steps• Stop capture, apply and propagation• Perform point in time recovery• Start apply, propagation and capture• Start SCN is determinated automatically

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Combined capture and apply

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Automatic split & merge

• Are split & merge procedures needed with combined capture and apply?

• Split of broken stream procedure is required:– to keep replicas in archive log window– to reduce number of LCRs to be re-captured by

capture process

• Start SCN = min {SCN app1, SCN app2, …}• All recaptured LCRs will be propagated to

all destinations, but applied only by one.

CERN IT Department

CH-1211 Genève 23


it

InternetServices



• Replacement of our custom scripts• Three ways of performing split and merge

operation– SQL script generation for manual execution– Splitting and merging procedures

• DBMS_STREAMS_ADM.SPLIT_STREAMS• DBMS_STREAMS_ADM.MERGE_STREAMS

– Automatic replication management (new in 11.2)• New capture parameters

– split_threshold– merge_theshold

• Monitor job running periodically

CERN IT Department

CH-1211 Genève 23


it

InternetServices



• DBA_STREAMS_SPLIT_MERGE

• DBA_STREAMS_SPLIT_MERGE_HIST

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Compare and converge

• DBMS_COMPARSION– Shared object synchronization package

• Compare rows in an object at 2 database• Converge objects in case of differences

– Supported database object types• Tables• Single-tables views• Materialized views• Synonyms for tables, single-tables views and

materialized views

– Version supporting:• Local >= 11.1• Remote >=10.1.0

CERN IT Department

CH-1211 Genève 23


it

InternetServices



– Comparison against different object types• e.g. table vs. materialized view• Column level synchronization

– Scan options• Identifying row differences (in separate table)• Random portion of data

– Percentage of data to be scanned– Cyclical way

• Custom portion of data (rowid range)

– No support for:• LONG, LONG RAW, ROWID, CLOB, NCLOB, BLOB,

BFILE• User defined types

CERN IT Department

CH-1211 Genève 23


it

InternetServices



DBA_COMPARISON General information about comparsions in the local databases

DBA_COMPARISON_COLUMNS Information about columns compared by each comparsion in local database

DBA_COMPARISON_SCAN Information about each scan run result

DBA_COMPARISON_SCAN_VALUES Information about range of rows compared during scan

DBA_COMPARISON_ROW_DIF Detailed information about row differences found in the each scan run. Populated when compare run with perform_row_dif parameter.

CERN IT Department

CH-1211 Genève 23


it

InternetServices



• Compare & converge steps– Create an comparison on object

• DBMS_COMPARISON.CREATE_COMPARISON

– Perform comparison• DBMS_COMPARISON.COMPARE

– If necessary do the conversion• DBMS_COMPARISON.CONVERGE

– Local wins– Remote wins

CERN IT Department

CH-1211 Genève 23


it

InternetServices



CERN IT Department

CH-1211 Genève 23


it

InternetServices



CERN IT Department

CH-1211 Genève 23


it

InternetServices



• Test result for artificial data with data set of 20 x varchar2 columns 17B each– Comparisons of

• consistent data– 18 MB/s• inconsistent data – 1,7 MB/s• purged table – 6 MB/s

– Convergence of• inconsistent data – 4 MB/s• purged table – 3,5 MB/s

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Streams performance advisor

• DBMS_STREAMS_ADVISOR_ADM– Analyzes local and remote (via dblink)

database configuration– Provides

• Oracle streams topology representation– By assignment of to each stream path

• Streams performance statistics– Component level– Session level

– Gathers streams related metrics with snapshots and compares them

• DBMS_STREAMS_ADVISOR_ADM.ANALYZE_CURRENT_PERFORMANCE;

– Stores statistics in set of dictionary views

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Streams performance advisor dictionary views

DBA_STREAMS_TP_COMPONENT contains information about each Oracle Streams component at each database

DBA_STREAMS_TP_COMPONENT_LINK contains information about how messages flow between Oracle Streams components

DBA_STREAMS_TP_COMPONENT_STAT contains temporary performance statistics and session statistics about each Oracle Streams component.

DBA_STREAMS_TP_DATABASE contains basic information about each database that contains Oracle Streams components (name, version, last advisor run etc.)

DBA_STREAMS_TP_PATH_BOTTLENECK contains temporary information about Oracle Streams components that might be slowing down the flow of messages in a stream path

http://download.oracle.com/docs/cd/B28359_01/server.111/b28320/statviews_5015.htm





CERN IT Department

CH-1211 Genève 23


it

InternetServices



• UTL_SPADV Package– Replacement of strmmon command line tool– Monitoring of whole stream path– Live streams data collector– Implemented with pl/sql procedures

• Stores measurements in db tables

– Monitoring management• UTL_SPADV.START_MONITORING• UTL_SPADV.STOP_MONITORING• UTL_SPADV.SHOW_STATS

CERN IT Department

CH-1211 Genève 23


it

InternetServices



• UTL_SPADV Package– Pre run actions

• @utlspadv.sql• grant execute on SYS.DBMS_LOCK to …

– Example outputPATH 1 RUN_ID 3 RUN_TIME 2008-MAR-20 10:52:22 CCA Y |<C> CAPTURE_HNS=>HUB.EXAMPLE.COM 10267 10040 3 LMR 95% 0% 3.3% "" LMP (1) 86.7% 0% 11.7% "" LMB 86.7% 0% 11.7% "" CAP+PS 10040 1.46E+06 3 16.7% 71.7% 11.7% "" |<PR> SPOKE1.EXAMPLE.COM=> 98.5% 0% 0.6% "" |<Q> "STRMADMIN"."DESTINATION_SPOKE1" 25 0 0 |<A> APPLY_SPOKE1 10042 100 4 APR 93.3% 0% 6.7% "" APC 98.1% 0% 1.8% "" APS (1) 953 272 76.7% "CPU + Wait for CPU" |<B> "NO BOTTLENECK IDENTIFIED"

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Other features

• LCRs tracking– By adding a tag to each captured LCR– V$STREAMS_MESSAGE_TRACKING

• Synchronous Capture– Uses internal mechanism for capturing

changes (DML only) instead of logminer

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Other features

• Oracle Streams Jobs Use Oracle Scheduler instead of DBMS_JOB

• Automated Alerts for Oracle Streams Clients and Thresholds in EM

• Support for – Transparent Data Encryption– XMLType Columns– Virtual Columns– Compressed table (in 11.2)

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Bugs discovered

• Propagation view population– After recreation of processes all propagation

metrics in v$propagation_sender– Assigned to the Oracle development

• Unable to split a stream as strmadmin user– ORA-02289: sequence does not exist– Fixed by: grant select on STREAMS$_SM_ID to

strmadmin

CERN IT Department

CH-1211 Genève 23


it

InternetServices


Conclusions

• Real time replication– About 10 times faster than 10g– Reduced resource consumption

• Easier streams recovery– Split & merge– Point in time recovery– Compare & converge synchronization

• More diagnostic and monitoring tools– Enterprise Manager– Streams Performance Advisor– LCRs tracking

Date post:	23-Dec-2015
Category:	Documents
Upload:	avis-short
View:	217 times
Download:	2 times

CERN IT Department CH-1211 Genève 23 Switzerland t Streams new features in 11g Zbigniew...

Documents