Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
What’sNewinOracleDataPump?OracleDatabase12cRelease2
RoyFSwongerVicePresident,DatabaseUpgrade&UtilitiesDecember6,2016
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.
2
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
What’sNewinDataPump?
3
ParallelExport/ImportofMetadataSubstitutionVariables&WildcardsREMAP_DIRECTORYLongIdentifiersupportTRUST_EXISTING_TABLE_PARTITIONSValidation&VerificationoptionsOther12.2FeaturesRecapof12.1Features
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ParallelMetadataExport:HowitUSEDTOWork
• StartwithESTIMATEphase– Gathertabledataobjects–Otherworkersremainidleuntildataobjectsaregathered
• Metadataexportedserially• Dataexportedinparallel
4
Pre-12.2
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ParallelMetadataExport:HowitWorksNow
• StartwithAnalysisstep–Metadataobjectspassedimmediatelytoworkersastheyarefound– E.g.Worker1findsasetofTABLEdefinitions,theyarehandedofftoworker2
• ESTIMATEphasestillhappens,butmetadatanolongerheldupbyestimate• Notes:–Worksfordumpfile jobs,andfornetworkjobsifdestinationdatabaseis12.2– Transportablejobsarenot(yet)parallelformetadata– ESTIMATEphasenowusesSTATISTICSonly– Restartworksasalways
5
NewFeaturein12.2
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ParallelMetadataExport:Logfile• 12.1.0.2 • 12.2.0.1
6
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
• Pre-12.2:–Oneworkerperpartition/subpartition– PQusedifpartitionsarelargeenough– Packagebodiesloadedinparallel
• Withpatchforbug22273229– Indexesbuiltinparallel– Constraintscreatedinparallel– Availableasbackportto12.1.0.2,11.2.0.4
• Startingwith12.2– Addedparallelimportofmost othermetadataobjects– Someexceptions• Types(duetoinheritance)• Schemas• Proceduralactions
7
ParallelMetadataImport
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ParallelMetadataImport:Internals• MetadataisexportedinXMLdocuments– EachXMLdocumentindumpfile containsn objectsofagiventype
• XMLdocumentsareallocatedtoworkers1documentatatime• Example:161userstoimport– Usersareexportedwithupto80usersperXMLdocument– WhathappenswithPARALLEL=8?
• Notes:–Worksforconventional(dumpfile)jobs– Not(yet)fortransportablejobsornetworkmode– Restartworkssameasalways– Statuscommandwillshowmultipleworkersonmetadata
8
XMLDoc1User1User2…User80
XMLDoc2User81User82…User160
XMLDoc1User161
Worker1
Worker2
Worker3
(Workers8-nwouldbeidle)
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ParallelMetadataImport:Logfile• ComparisonwithPARALLEL=8 for27586objectgrantsandMETRICS=Y– 12.1.0.2
– 12.2.0.1
9
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Performance:ParallelMetadataImport• ExamplesfromE-BusinessSuitetestdatabase
10
ObjectType Count 11.2.0.4PARALLEL=32
12.1.0.2PARALLEL=8
12.1.0.2PARALLEL=32With Patch
12.1.0.2PARALLEL=8
12.2.0.1PARALLEL=32
Comments
OBJECT_GRANT(owner)
27586 49 50 51 10 22 Hardconnectforeachgrant
SYNONYM 43254 105 109 111 25 44
TYPE 4364 108 114 119 111 110 Handledbysingleworker
PROCACT_SCHEMA 606 198 216 214 152 175 Handledbysingleworker
TABLE 33164 923 1160 1298 368 248
OBJECT_GRANT(table)
358649 541 543 578 142 157 Hardconnectforeachgrant
INDEX 53190 6721 5770 360 418 272
PACKAGE 53217 424 476 474 114 54
VIEW 34690 538 583 593 151 184
PACKAGE_BODY 52092 1363 1974 1186 1981 959 Always parallelsince11.2
*importtimeinseconds
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
SubstitutionVariablesforDumpfile Name• Substitutionvariablesfordumpfile name:– Pre-12.2:%Ugeneratesafixed-width2-digitnumber• e.g.dumpfile=exp%U.dmp
– Newoptionfor12.2expdp orimpdp:• %lor%L:Incrementingnumberfrom01upto2147483646
– Newoptionsin12.2expdp only:• %dor%D:DayofMonthinDDformat• %mor%M:NumberofMonthinMMformat• %yor%Y:YearinYYYYformat• %tor%T:FulldateinYYYYMMDDformat
11
$ expdp system/oracle directory=mydir \filesize=50K dumpfile=exp%T_%L.dmp full=y
...
...
...
. . exported "WMSYS"."WM$METADATA_MAP" 0 KB 0 rowsMaster table "SYSTEM"."SYS_EXPORT_FULL_01" successfully loaded/unloaded******************************************************************************Dump file set for SYSTEM.SYS_EXPORT_FULL_01 is: /home/oracle/exp20160917_01.dmp /home/oracle/exp20160917_02.dmp /home/oracle/exp20160917_03.dmp....../home/oracle/exp20160917_67.dmp /home/oracle/exp20160917_68.dmp /home/oracle/exp20160917_69.dmp /home/oracle/exp20160917_70.dmp /home/oracle/exp20160917_71.dmp /home/oracle/exp20160917_72.dmpJob "SYSTEM"."SYS_EXPORT_FULL_01" successfully completed at Sat Sep 17 23:47:31 2016 elapsed 0 00:03:00
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
WildcardsforTRANSPORT_DATAFILES• Useawilcard in12.2insteadoflistingeveryfile• Pre-12.2:
• NewFeature:wildcards– *(asterisk)matchesmultiplecharacters– ?(questionmark)matchesasinglecharacter
12
TRANSPORT_DATAFILES=users01.dbfTRANSPORT_DATAFILES=users02.dbf…TRANSPORT_DATAFILES=data1.dbfTRANSPORT_DATAFILES=data2.dbf…
TRANSPORT_DATAFILES=users*.dbfTRANSPORT_DATAFILES=data?.dbf…
$ impdp system/oracle@pdb2 network_link=sourcedb \version=12 full=y transportable=always metrics=y \exclude=statistics \directory=mydir \logfile=pdb2.log \transport_datafiles='/u02/oradata/CDB2/pdb2/user*.dbf’
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
REMAP_DIRECTORY• AppliestoDDLwheredirectoryspecsareused– E.g.CREATETABLESPACE
• Changedirectoryspecwithout changingfilenames• UsefulwhenmovingbetweenOSplatforms– Example:importingdumpfile createdonOpenVMSintodatabaseonLinuxREMAP_DIRECTORY="'DB1$:[HRDATA.PAYROLL]':'/db1/hrdata/payroll/'"
13
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
LongIdentifierSupport• Long(128-byte)identifiersaresupportedin12.2by– DataPumpexpdp/impdp– SQL*Loader–ORACLE_LOADERaccessdriver–ORACLE_DATAPUMPaccessdriver– ExternalTables
• Importingdatabasemustsupport128-byteidentifiers– BecarefulwhenexportingwithVERSION=12.1orearlier
14
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
TRUST_EXISTING_TABLE_PARTITIONS• Pre-12.2– Importingintoexistingtablewasdoneserially– DataPumpcouldn’tbesurethatpartitioninginDBmatchedpartitioningindumpfile
• New12.2Parameter:DATA_OPTIONS=TRUST_EXISTING_TABLE_PARTITIONS– Bigperformanceboost– Ifpartitionsdon’tmatch…error:
15
ORA-31693: Table data object "SH"."SALES_BIG_PT":"SALES_2000" failed to load/unload and is being skipped due to error:ORA-29913: error in executing ODCIEXTTABLEFETCH calloutORA-14401: inserted partition key is outside specified partition
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DataValidation&Verification
• DATA_OPTIONS=VALIDATE_TABLE_DATA– Importonly– Validatesdateandnumberformatsoftabledata– Defaultisnovalidation
• DATA_OPTIONS=VERIFY_STREAM_FORMAT– Exportonly– Defaultisnoverification
16
ExtraValidationforThingsThatShouldNeverHappen
Starting "SCOTT"."SYS_EXPORT_TABLE_01": scott/******** tables=t directory=dmpdir dumpfile=t.dmp reuse_dumpfiles=true Processing object type TABLE_EXPORT/TABLE/TABLE_DATAProcessing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICSProcessing object type TABLE_EXPORT/TABLE/STATISTICS/MARKERProcessing object type TABLE_EXPORT/TABLE/TABLE. . exported "SCOTT"."T" 5.570 KB 1 rowsORA-31694: master table "SCOTT"."SYS_EXPORT_TABLE_01" failed to load/unloadORA-02354: error in exporting/importing dataORA-26009: stream verification error: [1], [0], [0], [0]
ORA-02374: conversion error loading table "DPV"."TEST18"ORA-12899: value too large for column C1 (actual: 500, maximum: 498)ORA-02372: data for row: C8 : '
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Other12.2Features(1)• Usedirectpathloadinnetworkmode!– SpecifyACCESS_METHOD=DIRECT_PATH withNETWORK_LINK=<dblink>– AllowsnetworkimportofLONG andLONG RAW
• DataPumpavailableinInstantClient– ToolspackageforInstantClient– IncludesSQL*Loader,expdp,impdp,exp,imp
• Viewsthatdescribeavailabletransforms– DBMS_METADATA_TRANSFORMS– DBMS_METADATA_TRANSFORM_PARAMS– DBMS_METADATA_PARSE_ITEMS
17
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Other12.2Features(2)• Newinteractivecommands– TRACE parametercanbesetforarunningjob• Noneedtostop/restartjobfortracingtotakeeffect
– STOP_WORKER command• Killanindividualworkeryoubelievetobehungorstuck
– BothwillbedocumentedinMOSnotes
• Enhancedlogfiles–WhenMETRICS=Y• ShowworkerIDforeachitemprocessed• Showaccessmethodforeachtable
– Includecontentsofparfile inlogfile
18
18-SEP-16 15:24:30.950: ;;; **********************************************************18-SEP-16 15:24:30.951: ;;; Parfile values:18-SEP-16 15:24:30.953: ;;; parfile: job_name=md_exp_16_1220118-SEP-16 15:24:30.955: ;;; parfile: reuse_dumpfiles=Y18-SEP-16 15:24:30.957: ;;; parfile: logtime=all18-SEP-16 15:24:30.958: ;;; parfile: metrics=Y18-SEP-16 15:24:30.960: ;;; parfile: parallel=1618-SEP-16 15:24:30.962: ;;; parfile: full=Y18-SEP-16 15:24:30.963: ;;; parfile: logfile=md_exp_16_12201.log18-SEP-16 15:24:30.965: ;;; parfile: dumpfile=md16_12201_%U.dmp18-SEP-16 15:24:30.966: ;;; parfile: directory=EBSIMP18-SEP-16 15:24:30.968: ;;; *********************************************************
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
Recapof12.1Features• VIEWS_AS_TABLES parameter– Letsyouexportthecontentsofaviewasatable
• TRANSFORM parameteroptions– TRANSFORM=DISABLE_ARCHIVE_LOGGING:Y• Willdisablearchiveloggingduringimportfortablesand/orindexes
– TRANSFORM=LOB_STORAGE:SECUREFILE– TRANSFORM=STORAGE:N– TRANSFORM=TABLE_COMPRESSION:<compression_clause>
• LOGTIME=[NONE |STATUS |LOGFILE |ALL ]parameter–Willwritetimestampsonstatusand/orlogfile messages
19
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DataPumpNewsinOracle12c• TRANSFORM optiontoenableAdvanced/HCCCompression– Example:• TRANSFORM=TABLE_COMPRESSION:"compress for query high"
– But: Granularityonlyontheentireimport• Workarounds:– Precreateobjects• Downside:Willslowdownimport!!!or:
– PrecreatethetablespacewithCOMPRESSoption• create tablespace ARCHIGH datafile 'archigh.ora' size 100G default compress for archive high;
– ThenrunDataPumpwithTRANSFORM=TABLE_COMPRESSION:N• Thiswilldropallembeddedcompressionattributesassociatedwiththetables• Nowtablespacecompressionoptionwillbeusedforallnewlycreatedtables
20
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
ExportingViewsasTables
TABLE1
VIEW1 expdp system/mgrviews_as_tables=scott.view1 ...
impdp system/mgrremap_table=view1:scott.table1 ...
TABLE1
21
Copyright©2016Oracle and/oritsaffiliates.Allrightsreserved.|
ImprovedLogFiles:METRICS=Y andLOGTIME=ALL
22
METRICS=Y(Added 11.2)
BasicLogfile
LOGTIME=ALL(Added 12.1)
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
EnhancedCompressionAlgorithm• COMPRESSION_ALGORITHM– Definesthecompressionalgorithmwhencompressingdumpfiles
• BASIC Thesamealgorithmusedinpreviousversions.Goodcompression,withoutseverelyimpactingonperformance• LOW: ForusewhenreducedCPUutilizationisapriorityovercompressionratio• MEDIUM: Recommendedoption.SimilarcharacteristicstoBASIC,butusesadifferentalgorithm• HIGH: Maximumavailablecompression,butmoreCPUintensive
– Performance:• Compressionratio• CPUusage
– RequiresAdvancedCompressionOptionlicense
$ expdp scott/tiger tables=emp directory=mydir dumpfile=emp.dmp logfile=expdp_emp.log compression=all compression_algorithm=medium
23
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
EnhancedCompressionAlgorithm• Customerevaluation– BASICat3.5TB/hour
– MEDIUMat7.0TB/hour
2x
24
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
DownloadSlidesfromtheDatabaseUpgradeBlog• http://blogs.oracle.com/UPGRADE
25
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.| 26