Optimizing Table Scans in the Cloud - Oracle User GroupPartitioning Confidential –Oracle...

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|

OptimizingTableScansintheCloud

JohnClarkeSoftwareDevelopmentDirectorReal-WorldPerformanceServerTechnologies

Confidential– OracleInternal/Restricted/HighlyRestricted


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Confidential– OracleInternal/Restricted/HighlyRestricted 3


BridgingtheDividefromToday’sPerformancetoWhatisPossibleWhatisReal-WorldPerformancein2018?


Real-WorldPerformance2018

• PartoftheDatabaseDevelopmentOrganization• GlobalTeamlocatedinUSA,Europe,Asia• 350+combinedyearsofOracledatabaseexperience• InnovatetoachieveexceptionalDatabasePerformance• Ourmethods:

• Usetheproductasitwasdesignedtobeused• Numericalandlogicaldebuggingtechniques• Educateothersaboutthebestperformancemethodsandtechniques• Avoidandeliminate“tuning”byhacking/guessing/luck

WhoWeAre


We’veBeenHereBefore

• Howmanyrowsdoyouneedtofind?a) Oneb) Afewc) Alotd) Idon’tknow

• Doyouscanoruseanindex?• Ifyoudon’tknow,whataccessmethodistheleastrisky?



TheQuestionWe’reAsking


SELECT d_year, d_sellingseason, c_region, SUM(lo_extendedprice),SUM(lo_supplycost)FROM lineorder

JOIN customer ON lo_custkey = c_custkeyJOIN date_dim ON lo_orderdate = d_datekeyJOIN part ON lo_partkey = p_partkeyJOIN supplier ON lo_suppkey = s_suppkey

WHERE d_month IN (’June’,’July’,’August’)AND p_mfgr IN (’MFGR#1’,’MFGR#2’)AND s_nation = ’China’

GROUP BY d_year, d_sellingseason, c_regionORDER BY d_year, d_sellingseason, c_region

“Showmethepriceandcostbyyear,sellingseason,andcustomerregionforallgoodssoldinJune,July,andAugustforpartsmanufacturedbyMFGR#1

andMFGR#2inChina”


OracleCloudInfrastructure

• OracleDatabaseCloudService– OracleCloudInfrastructure• OracleExadataCloudService- OracleCloudInfrastructure

WhereWe’reAskingtheQuestion



Agenda

WhyTableScans?

MakingScansSmaller

Where’sOurLeverage?

RearrangingtheData

ThingsWeCanDotoSpeedUpNextOperationinPlan

1

2

3

4

5



Agenda

WhyTableScans?

MakingScansSmaller


RearrangingtheData


1

2

3

4

5



WhyTableScans?

• Ourqueryjoins4dimensiontablestoa1-billionrowfacttable• WouldyouexpectthisquerytobeI/Obound?• Ifweuseindexes:– Firstjointofacttableretrieves42millionrows,or4.2%– Aftercompletingadditionaljoins,weendupdoing~320millionrandomreads

• 320mrandomreads@5ms/read=~1600seconds– Is5msforarandomI/Oan“oldtechnumber”?– Ifthe“newtechnumbernumber”is1ms,we’relookingat~5minutesforI/O

SomeHistoryandMath



WhyTableScans?

• Wespend43seconds onI/O,not26minutesor5minutes• Ouraveragerandomreadistakingafractionofamillisecond• We’reCPU-bound,notI/Obound

IndexAccess



WhyTableScans?

• Whataboutafulltablescan?• Wehaveabout14millionblocks• The“oldtechnumber”formulti-blockreadsisabout6or7msperMBR• Amulti-blockreadcountof128=109kmulti-blockreads• 109kmulti-blockreadsat6.5msperMBRmeanswe’dspendunderaseconddoingI/O• Let’ssee…

FullTableScans



WhyTableScans?

• We375msdoingI/OwithanaverageI/Osizeof25MB• “Effective”MBRCmuchhigherthan128• We’restillCPU-bound

FullTableScans



WhyTableScans?

• InOracle’sCloud,randomandsequentialreadsaremuchfasterthantheoldnumberspeoplethinkabout• Forbothindexandtablescanaccess,thequeriesareCPU-bound,notI/Obound• Inthiscase,scanswere6xfaster.Wasthisbecauseofscans,joins,aggregation,orsomethingelse?• Timetodigabitdeeper!

TheNewMath&WhatWe’veLearned



WhyTableScans?LeverageMatrix


Method %of TimeinDataAcquisition

%of TimeinJoins %of TimeinSort/Aggregate

Elapsedseconds

ScanswithHashJoins

86% 12.24% 1.60% 187

IndexaccessandNLJoin

99% .08% .17% 1,195


WhyTableScans?NumbersSoFar



Agenda

WhyTableScans?

MakingScansSmaller


RearrangingtheData


1

2

3

4

5



MakingScansSmaller

• Partitioning isameanstoprunedataandreduceI/O&CPU• CompressionisameanstoreducesizeofdataondiskandreduceI/O• OracleCloudInfrastructuresupportsHybridColumnarCompression• Let’stryitout

PartitioningandCompression



MakingScansSmallerPartitioning


StillCPU-bound,butusesalotlessCPUthanscanswithout

partitioning


MakingScansSmallerPartitioning


AlthoughCPU-bound,I/ObytesisaproxyforCPUconsumption

TableScanswithout

partitioning

TableScanswithpartitioning

Wepartitionedondatejoinkeyandourpredicatesfiltered¾ofthe

data


MakingScansSmaller

• Acommonproblemweseeisover-partitioning– Proxyforindexes?– Usedtoavoidcontention?

• Toomanypartitionscancausemanyproblems:– Excessivetimeduringparse&execute– Highmetadatacost– DDLmoreexpensiveduetodatadictionaryoverheads– ExacerbatedwithRAC– Problemscouldrevealthemselvesinnon-obviousways

PartitioningWarnings



MakingScansSmallerHCCCompressionwithScansandPartitioning,OracleDatabaseCloudService


QueryisCPU-boundagainbutonlyranmarginallyfaster

Without HCC


MakingScansSmallerHCCCompressionwithScansandPartitioning,OracleDatabaseCloudService


Wescanned3.5xlessdatabutonlyimprovedqueryperformanceby

35%

Without HCC


MakingScansSmaller

• Weweren’tI/Oboundtobeginwith,wewereCPU-bound• CPU&timetoparseHCCblockslessthantimetoparseuncompressedblocks,but…• WeneedCPUtodecompresscompresseddata• Queryingsmallerdatasetsdoesn’tyieldlinearperformancegains• Conventionalmindsetvs.moderncapabilities

HCCCompressionwithScansandPartitioning,OracleDatabaseCloudService



MakingScansSmallerLeverageMatrix



%of TimeinJoins

%of TimeinSort/Aggregate

ElapsedSeconds

PartitioningwithCompression(OCIDBCS) 81% 14.5% 5.8% 36

Partitioningon(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins(OCI DBCS) 86% 12.24% 1.60% 187

IndexaccessandNLJoin(OCIDBCS) 99% .08% .17% 1,195


MakingScansSmallerNumbersSoFar



Agenda

WhyTableScans?

MakingScansSmaller


RearrangingtheData

ThingsWeCanDotoSpeedUpNextOperations`inPlan

1

2

3

4

5



Where’sOurLeverage?SomeProfilingData


IndexScans TableScans

Mostofourtimeisinparsingrows/blocks


Where’sOurLeverage?ProfilingDatawithHCCandPartitioningonOracleDatabaseCloudService


Timespentparsingcolumns/rowsinHCCformat



• Mostofourtimeisbeingspentondataacquisition• DataacquisitionisCPU-boundonOracleDBCloudService• Profilingshowsit’slargelyrelatedtoparsingblocks• Whatifwecouldrecruitmoreresourcesforblockparsing,parseblocksinparallel,offloadthisworktodifferentmachines,anddecreasewallclocktime?• Dowehaveanytechnologythatdoesthis?

BlockParsing



Where’sOurLeverage?ExadataCloudService


• QuerystillCPU-boundbutwesee7.5%(2seconds)onI/O• RememberwesawnoI/OonDBCS

• I/OinthiscasemeansanythingintheI/Opath,includingCPUonstoragecells.4.35%ofourtimeison“smartscan”



• 4.35%of13seconds=~.5seconds,multiplyby2slaves=~1 seconcells• OnExadataweuseupto10parallelrequestsperslave– 2slaves=up20parallelrequestspercell– 7cells=~140 parallelrequestsintotal,whichis70xmorethanDWCS

• InBMCwespend=~70 CPUsecondsonscan• Offloadingallowsustoparseblocksinparallel,reduceelapsedtime,andreducecomputenodeCPUtime• Bonusquestion– ifwedo10requestsperslavewith1MBI/Osize,whatisshouldourminimumpartitionsizebe?

ExadataCloudService toOffloadBlockParsing




Didyounoticewe’restillspending24.5CPUsecondsnotdoingthescans?

ExadataCloudService toOffloadBlockParsing



Where’sOurLeverage?ExadataCloudService toOffloadBlockParsingComputenodeProfiling


Timeoncomputenodenolongerdominatedbyparsingblocks


Where’sOurLeverage?LeverageMatrix



%of TimeinJoins


ElapsedSeconds

PartitioningwithCompression(OCIExaCS) 48% 51% 1% 13

PartitioningwithCompression (OCIDBCS) 81% 14.5% 5.8% 36

PartitioningonBMC(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins (OCIDBCS) 86% 12.24% 1.60% 187




• Exadataprovidessomeotherinterestingalternativestoexplore• ZoneMapswithAttributeClusteringprovideandadditionalmeanstopruneI/OandreduceCPU• Inadditiontopartitioningonourdatedimension’sjoinkey,let’simplementAttributeClusteringwithaZoneMaponourSupplierdimension’sjoinkey

MoreYouCanDoonExadata



Where’sOurLeverage?ExadataCloudService withClusteringandZoneMaps


• Elapsedtimereducedfrom13secondsto5seconds• ComputenodeCPUreducedfrom24.5to6.7seconds• Likepartitioning,ZoneMapswithClusteringmeansfewer

calls toExadata,witheachcallbeingmore“row-rich”


Where’sOurLeverage?LeverageMatrix



%of TimeinJoins


ElapsedSeconds

PartitioningwithHCC&ZoneMaps(OCIExaCS)

57% 29% 14% 5

PartitioningwithHCC(OCIExaCS) 48% 51% 1% 13


Partitioning(OCIDBCS) 79% 17% 4% 55

ScanswithHashJoins (OCIDBCS) 86% 12.24% 1.60% 187



Where’sOurLeverage?NumbersSoFar



Agenda

WhyTableScans?

MakingScansSmaller


RearrangingtheData


1

2

3

4

5



RearrangingtheData

• We’vedemonstratedthatparsingblocksconsumesCPUandcontributestoqueryelapsedtimeduringscans• OffloadingtoExadataprovidesusmoreCPUstoparseblocks• HowwouldIn-MemoryColumnarrepresentationimpactourresults?• Let’stestwithDatabaseIn-Memory

In-MemoryColumnar



RearrangingtheDataIn-MemoryColumnar


ScanwithPartitioningandIn-Memory(DBCloudService)

ScanwithPartitioning(DBCloudService,rowformat)


RearrangingtheDataIn-MemoryColumnar


ScanwithPartitioningand

DBIM


RearrangingtheDataIn-MemoryColumnarLeverageMatrix



%of TimeinJoins


ElapsedSeconds

DBIMwith Partitioning(OCIDBCS) 18% 64% 18% 13

PartitioningwithHCC&ZoneMaps(OCIExaCS) 57% 29% 14% 5




ScanswithHashJoins (OCI DBCS) 86% 12.24% 1.60% 187



RearrangingtheDataNumbersSoFar



Agenda

WhyTableScans?

MakingScansSmaller


RearrangingtheData

ThingsWeCanDoforJoinsandAggregation

1

2

3

4

5




• BloomFiltersprovidemeanstoefficientlyfilterdata,reducingthevolumeofdataforhashjoinsanddistributioninsubsequentplansteps• BloomFilterevaluationcanbepusheddowntoExadatasowecanleveragestoragecellCPUs• BloomFilterevaluationalsopusheddowntoIn-Memorycolumnstoreandabletousedifferent&moreefficientalgorithms• Let’stest

BloomFilters



ThingsWeCanDoforJoinsandsAggregationBloomFilters


ScanwithPartitioningand

DBIM

ScanwithBloomFilters,

PartitioningandDBIM


ThingsWeCanDoforJoinsandAggregationNumbersSoFarwithBloomFiltersonDBCSandExaCS



ThingsWeCanDoforJoinsandAggregationNumbersSoFarwithBloomFiltersonDBCSandExaCS




• Pushdownaggregationtoscan• In-MemoryAggregation performsaggregationduringscan• Let’senableitandtest

In-MemoryAggregation



ThingsWeCanDoforJoinsandAggregationElapsedSeconds



ThingsWeCanDoforJoinsandAggregationCPUSeconds



ThingsWeCanDoforJoinsandAggregationLeverageChart



%of TimeinJoins %of TimeinSort/Aggregate

ElapsedSeconds

IMCwithIMAandPartitioning(OCIDBCS) 100% 0% 0% 1

IMC&Partitioning withBFsandZoneMaps(OCIExaCS) 28% 58% 14% 3

Partitioning withBFsandZoneMaps(OCIExaCS) 33% 50% 17% 4

Partitioning withBFs(OCIExaCS) 30% 50% 20% 6

IMCwithBFsandPartitioning(OCIDBCS) 30% 40% 10% 5

IMCwith Partitioning(OCIDBCS) 18% 64% 18% 13

PartitioningwithHCC&ZoneMaps(OCIExaCS) 57% 29% 14% 5




ScanswithHashJoins ((OCIDBCS) 86% 12.24% 1.60% 187



Bonus

• Wecan’tuseZoneMapswithAttributeClusteringonnon-Exadata,buthere’ssomething(relatively)freeintheCloud• Let’smanuallysortthedatatoleverageIn-MemoryMin-Maxpruning

SortingonDatabaseCloudService,In-Memory,In-MemoryAggregation


Lookfor”IMscanCUspruned”or“IMscanrowsoptimized”stats


FeaturesAvailability

Feature Oracle CloudInfrastructureDatabaseCloudService

Oracle CloudInfrastructureExadataCloudService

Partitioning

Hybrid ColumnarCompression

ZoneMapsand AttributeClustering

In-MemoryandIn-Memory Aggregation

BloomFilters

57


Summary

• Thesedays,scanperformanceusuallyisn’taboutreducingI/O,it’saboutreducingCPU• Thereareanumberofwaystodothis• WereducedCPUfrom1,176secondsto1.7secondsforthesamequery• Wereducedelapsedtimefrom1,195secondsto1secondforthesamequery• Lookforleverage!• Don’tsettlefor“goodenough”



Summary


Whatcanyoudowithyoursystem?


SafeHarborStatementTheprecedingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.


Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| Confidential– OracleInternal/Restricted/HighlyRestricted 61

Date post:	24-Jul-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Optimizing Table Scans in the Cloud - Oracle User GroupPartitioning Confidential –Oracle...

Documents