Optimizing Table Scans in the Cloud - Oracle User GroupPartitioning Confidential –Oracle...

transcript

OptimizingTableScansintheCloud

JohnClarkeSoftwareDevelopmentDirectorReal-WorldPerformanceServerTechnologies

Confidential– OracleInternal/Restricted/HighlyRestricted

SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Confidential– OracleInternal/Restricted/HighlyRestricted 3

BridgingtheDividefromToday’sPerformancetoWhatisPossibleWhatisReal-WorldPerformancein2018?

Real-WorldPerformance2018

• PartoftheDatabaseDevelopmentOrganization• GlobalTeamlocatedinUSA,Europe,Asia• 350+combinedyearsofOracledatabaseexperience• InnovatetoachieveexceptionalDatabasePerformance• Ourmethods:

• Usetheproductasitwasdesignedtobeused• Numericalandlogicaldebuggingtechniques• Educateothersaboutthebestperformancemethodsandtechniques• Avoidandeliminate“tuning”byhacking/guessing/luck

WhoWeAre

We’veBeenHereBefore

• Howmanyrowsdoyouneedtofind?a) Oneb) Afewc) Alotd) Idon’tknow

• Doyouscanoruseanindex?• Ifyoudon’tknow,whataccessmethodistheleastrisky?

TheQuestionWe’reAsking

SELECT d_year, d_sellingseason, c_region, SUM(lo_extendedprice),SUM(lo_supplycost)FROM lineorder

JOIN customer ON lo_custkey = c_custkeyJOIN date_dim ON lo_orderdate = d_datekeyJOIN part ON lo_partkey = p_partkeyJOIN supplier ON lo_suppkey = s_suppkey

WHERE d_month IN (’June’,’July’,’August’)AND p_mfgr IN (’MFGR#1’,’MFGR#2’)AND s_nation = ’China’

GROUP BY d_year, d_sellingseason, c_regionORDER BY d_year, d_sellingseason, c_region

“Showmethepriceandcostbyyear,sellingseason,andcustomerregionforallgoodssoldinJune,July,andAugustforpartsmanufacturedbyMFGR#1

andMFGR#2inChina”

OracleCloudInfrastructure

• OracleDatabaseCloudService– OracleCloudInfrastructure• OracleExadataCloudService- OracleCloudInfrastructure

WhereWe’reAskingtheQuestion

Agenda

WhyTableScans?

MakingScansSmaller

Where’sOurLeverage?

RearrangingtheData

ThingsWeCanDotoSpeedUpNextOperationinPlan

Agenda

WhyTableScans?

MakingScansSmaller

RearrangingtheData

WhyTableScans?

• Ourqueryjoins4dimensiontablestoa1-billionrowfacttable• WouldyouexpectthisquerytobeI/Obound?• Ifweuseindexes:– Firstjointofacttableretrieves42millionrows,or4.2%– Aftercompletingadditionaljoins,weendupdoing~320millionrandomreads

• 320mrandomreads@5ms/read=~1600seconds– Is5msforarandomI/Oan“oldtechnumber”?– Ifthe“newtechnumbernumber”is1ms,we’relookingat~5minutesforI/O

SomeHistoryandMath

WhyTableScans?

• Wespend43seconds onI/O,not26minutesor5minutes• Ouraveragerandomreadistakingafractionofamillisecond• We’reCPU-bound,notI/Obound

IndexAccess

WhyTableScans?

• Whataboutafulltablescan?• Wehaveabout14millionblocks• The“oldtechnumber”formulti-blockreadsisabout6or7msperMBR• Amulti-blockreadcountof128=109kmulti-blockreads• 109kmulti-blockreadsat6.5msperMBRmeanswe’dspendunderaseconddoingI/O• Let’ssee…

FullTableScans

WhyTableScans?

• We375msdoingI/OwithanaverageI/Osizeof25MB• “Effective”MBRCmuchhigherthan128• We’restillCPU-bound

FullTableScans

WhyTableScans?

• InOracle’sCloud,randomandsequentialreadsaremuchfasterthantheoldnumberspeoplethinkabout• Forbothindexandtablescanaccess,thequeriesareCPU-bound,notI/Obound• Inthiscase,scanswere6xfaster.Wasthisbecauseofscans,joins,aggregation,orsomethingelse?• Timetodigabitdeeper!

TheNewMath&WhatWe’veLearned

WhyTableScans?LeverageMatrix

Method %of TimeinDataAcquisition

%of TimeinJoins %of TimeinSort/Aggregate

Elapsedseconds

ScanswithHashJoins

86% 12.24% 1.60% 187

IndexaccessandNLJoin

99% .08% .17% 1,195

WhyTableScans?NumbersSoFar

Agenda

WhyTableScans?

MakingScansSmaller

RearrangingtheData

MakingScansSmaller

• Partitioning isameanstoprunedataandreduceI/O&CPU• CompressionisameanstoreducesizeofdataondiskandreduceI/O• OracleCloudInfrastructuresupportsHybridColumnarCompression• Let’stryitout

PartitioningandCompression

MakingScansSmallerPartitioning

StillCPU-bound,butusesalotlessCPUthanscanswithout

partitioning

MakingScansSmallerPartitioning

AlthoughCPU-bound,I/ObytesisaproxyforCPUconsumption

TableScanswithout

partitioning

TableScanswithpartitioning

Wepartitionedondatejoinkeyandourpredicatesfiltered¾ofthe

MakingScansSmaller

• Acommonproblemweseeisover-partitioning– Proxyforindexes?– Usedtoavoidcontention?

• Toomanypartitionscancausemanyproblems:– Excessivetimeduringparse&execute– Highmetadatacost– DDLmoreexpensiveduetodatadictionaryoverheads– ExacerbatedwithRAC– Problemscouldrevealthemselvesinnon-obviousways

PartitioningWarnings

MakingScansSmallerHCCCompressionwithScansandPartitioning,OracleDatabaseCloudService

QueryisCPU-boundagainbutonlyranmarginallyfaster

Without HCC

MakingScansSmallerHCCCompressionwithScansandPartitioning,OracleDatabaseCloudService

Wescanned3.5xlessdatabutonlyimprovedqueryperformanceby

Without HCC

MakingScansSmaller

• Weweren’tI/Oboundtobeginwith,wewereCPU-bound• CPU&timetoparseHCCblockslessthantimetoparseuncompressedblocks,but…• WeneedCPUtodecompresscompresseddata• Queryingsmallerdatasetsdoesn’tyieldlinearperformancegains• Conventionalmindsetvs.moderncapabilities

HCCCompressionwithScansandPartitioning,OracleDatabaseCloudService

MakingScansSmallerLeverageMatrix

%of TimeinJoins

%of TimeinSort/Aggregate

ElapsedSeconds

PartitioningwithCompression(OCIDBCS) 81% 14.5% 5.8% 36

Partitioningon(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins(OCI DBCS) 86% 12.24% 1.60% 187

IndexaccessandNLJoin(OCIDBCS) 99% .08% .17% 1,195

MakingScansSmallerNumbersSoFar

Agenda

WhyTableScans?

MakingScansSmaller

RearrangingtheData

ThingsWeCanDotoSpeedUpNextOperations`inPlan

Where’sOurLeverage?SomeProfilingData

IndexScans TableScans

Mostofourtimeisinparsingrows/blocks

Where’sOurLeverage?ProfilingDatawithHCCandPartitioningonOracleDatabaseCloudService

Timespentparsingcolumns/rowsinHCCformat

• Mostofourtimeisbeingspentondataacquisition• DataacquisitionisCPU-boundonOracleDBCloudService• Profilingshowsit’slargelyrelatedtoparsingblocks• Whatifwecouldrecruitmoreresourcesforblockparsing,parseblocksinparallel,offloadthisworktodifferentmachines,anddecreasewallclocktime?• Dowehaveanytechnologythatdoesthis?

BlockParsing

Where’sOurLeverage?ExadataCloudService

• QuerystillCPU-boundbutwesee7.5%(2seconds)onI/O• RememberwesawnoI/OonDBCS

• I/OinthiscasemeansanythingintheI/Opath,includingCPUonstoragecells.4.35%ofourtimeison“smartscan”

• 4.35%of13seconds=~.5seconds,multiplyby2slaves=~1 seconcells• OnExadataweuseupto10parallelrequestsperslave– 2slaves=up20parallelrequestspercell– 7cells=~140 parallelrequestsintotal,whichis70xmorethanDWCS

• InBMCwespend=~70 CPUsecondsonscan• Offloadingallowsustoparseblocksinparallel,reduceelapsedtime,andreducecomputenodeCPUtime• Bonusquestion– ifwedo10requestsperslavewith1MBI/Osize,whatisshouldourminimumpartitionsizebe?

ExadataCloudService toOffloadBlockParsing

Didyounoticewe’restillspending24.5CPUsecondsnotdoingthescans?

ExadataCloudService toOffloadBlockParsing

Where’sOurLeverage?ExadataCloudService toOffloadBlockParsingComputenodeProfiling

Timeoncomputenodenolongerdominatedbyparsingblocks

Where’sOurLeverage?LeverageMatrix

%of TimeinJoins

ElapsedSeconds

PartitioningwithCompression(OCIExaCS) 48% 51% 1% 13

PartitioningwithCompression (OCIDBCS) 81% 14.5% 5.8% 36

PartitioningonBMC(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins (OCIDBCS) 86% 12.24% 1.60% 187

• Exadataprovidessomeotherinterestingalternativestoexplore• ZoneMapswithAttributeClusteringprovideandadditionalmeanstopruneI/OandreduceCPU• Inadditiontopartitioningonourdatedimension’sjoinkey,let’simplementAttributeClusteringwithaZoneMaponourSupplierdimension’sjoinkey

MoreYouCanDoonExadata

Where’sOurLeverage?ExadataCloudService withClusteringandZoneMaps

• Elapsedtimereducedfrom13secondsto5seconds• ComputenodeCPUreducedfrom24.5to6.7seconds• Likepartitioning,ZoneMapswithClusteringmeansfewer

calls toExadata,witheachcallbeingmore“row-rich”

Where’sOurLeverage?LeverageMatrix

%of TimeinJoins

ElapsedSeconds

PartitioningwithHCC&ZoneMaps(OCIExaCS)

57% 29% 14% 5

PartitioningwithHCC(OCIExaCS) 48% 51% 1% 13

Partitioning(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins (OCIDBCS) 86% 12.24% 1.60% 187

Where’sOurLeverage?NumbersSoFar

Agenda

WhyTableScans?

MakingScansSmaller

RearrangingtheData

• We’vedemonstratedthatparsingblocksconsumesCPUandcontributestoqueryelapsedtimeduringscans• OffloadingtoExadataprovidesusmoreCPUstoparseblocks• HowwouldIn-MemoryColumnarrepresentationimpactourresults?• Let’stestwithDatabaseIn-Memory

In-MemoryColumnar

RearrangingtheDataIn-MemoryColumnar

ScanwithPartitioningandIn-Memory(DBCloudService)

ScanwithPartitioning(DBCloudService,rowformat)

RearrangingtheDataIn-MemoryColumnar

ScanwithPartitioningand

RearrangingtheDataIn-MemoryColumnarLeverageMatrix

%of TimeinJoins

ElapsedSeconds

DBIMwith Partitioning(OCIDBCS) 18% 64% 18% 13

PartitioningwithHCC&ZoneMaps(OCIExaCS) 57% 29% 14% 5

ScanswithHashJoins (OCI DBCS) 86% 12.24% 1.60% 187

RearrangingtheDataNumbersSoFar

Agenda

WhyTableScans?

MakingScansSmaller

RearrangingtheData

ThingsWeCanDoforJoinsandAggregation

• BloomFiltersprovidemeanstoefficientlyfilterdata,reducingthevolumeofdataforhashjoinsanddistributioninsubsequentplansteps• BloomFilterevaluationcanbepusheddowntoExadatasowecanleveragestoragecellCPUs• BloomFilterevaluationalsopusheddowntoIn-Memorycolumnstoreandabletousedifferent&moreefficientalgorithms• Let’stest

BloomFilters

ThingsWeCanDoforJoinsandsAggregationBloomFilters

ScanwithPartitioningand

ScanwithBloomFilters,

PartitioningandDBIM

ThingsWeCanDoforJoinsandAggregationNumbersSoFarwithBloomFiltersonDBCSandExaCS

• Pushdownaggregationtoscan• In-MemoryAggregation performsaggregationduringscan• Let’senableitandtest

In-MemoryAggregation

ThingsWeCanDoforJoinsandAggregationElapsedSeconds

ThingsWeCanDoforJoinsandAggregationCPUSeconds

ThingsWeCanDoforJoinsandAggregationLeverageChart

%of TimeinJoins %of TimeinSort/Aggregate

ElapsedSeconds

IMCwithIMAandPartitioning(OCIDBCS) 100% 0% 0% 1

IMC&Partitioning withBFsandZoneMaps(OCIExaCS) 28% 58% 14% 3

Partitioning withBFsandZoneMaps(OCIExaCS) 33% 50% 17% 4

Partitioning withBFs(OCIExaCS) 30% 50% 20% 6

IMCwithBFsandPartitioning(OCIDBCS) 30% 40% 10% 5

IMCwith Partitioning(OCIDBCS) 18% 64% 18% 13

PartitioningwithHCC&ZoneMaps(OCIExaCS) 57% 29% 14% 5

ScanswithHashJoins ((OCIDBCS) 86% 12.24% 1.60% 187

• Wecan’tuseZoneMapswithAttributeClusteringonnon-Exadata,buthere’ssomething(relatively)freeintheCloud• Let’smanuallysortthedatatoleverageIn-MemoryMin-Maxpruning

SortingonDatabaseCloudService,In-Memory,In-MemoryAggregation

Lookfor”IMscanCUspruned”or“IMscanrowsoptimized”stats

FeaturesAvailability

Feature Oracle CloudInfrastructureDatabaseCloudService

Oracle CloudInfrastructureExadataCloudService

Partitioning

Hybrid ColumnarCompression

ZoneMapsand AttributeClustering

In-MemoryandIn-Memory Aggregation

BloomFilters

Summary

• Thesedays,scanperformanceusuallyisn’taboutreducingI/O,it’saboutreducingCPU• Thereareanumberofwaystodothis• WereducedCPUfrom1,176secondsto1.7secondsforthesamequery• Wereducedelapsedtimefrom1,195secondsto1secondforthesamequery• Lookforleverage!• Don’tsettlefor“goodenough”

Summary

Whatcanyoudowithyoursystem?

SafeHarborStatementTheprecedingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Optimizing Table Scans in the Cloud - Oracle User GroupPartitioning Confidential –Oracle...

Documents