A System for Automatic Recording and Prediction of Design Quality Metrics

A System for Automatic Recording and Prediction of Design Quality Metrics

Andrew B. Kahng and Stefanus Mantik*

UCSD CSE and ECE Depts., La Jolla, CA

*UCLA CS Dept., Los Angeles, CA

IntroductionIntroduction

Time-to-market window is shrinking very rapidly Product quality and design process must continually

improve Currently, there are no standards or infrastructure

for measuring and recording the semiconductor design process

METRICS provides Standard infrastructure for the collection and the

storage of design process information Standard list of design metrics and process metrics Analyses and reports that are useful for design

process optimization

METRICS allows: Collect, Data-Mine, Measure, Diagnose, then Improve

Related WorksRelated Works

Enterprise- and project-level metrics (“Numetrics Complexity Unit”) Numetrics Management Systems DPMS

Other in-house data collection systems e.g., TI (DAC 96 BOF), OxSigen LLC (Siemens Semiconductor)

Design Process Management [Jacome93, Brockman92, Johnson96]

Web-based design support IPSymphony, WELD, VELA, etc.

Continuous process improvement

Data mining and visualization

OutlineOutline

METRICS system architecture and standards

METRICS for design flow

Flow METRICS experiments

METRICS integration with datamining

Datamining integration experiments

Issues and conclusions

METRICS System ArchitectureMETRICS System Architecture

Inter/Intra-net

DBMetrics Data Warehouse

WebServer

JavaApplets

DataMining

Reporting

Transmitter Transmitterwrapper

Tool Tool Tool

TransmitterAPI

XML

Generic and Specific Tool MetricsGeneric and Specific Tool Metrics

tool_name stringtool_version stringtool_vendor stringcompiled_date mm/dd/yyyystart_time hh:mm:ssend_time hh:mm:sstool_user stringhost_name stringhost_id stringcpu_type stringos_name stringos_version stringcpu_time hh:mm:ss

Generic Tool Metricsnum_cells integernum_nets integerlayout_size doublerow_utilization doublewirelength doubleweighted_wl double

num_layers integernum_violations integernum_vias integerwirelength doublewrong-way_wl doublemax_congestion double

Placement Tool Metrics

Routing Tool Metrics

Partial list of metrics now being collected in Oracle8i

OutlineOutline

METRICS system architecture and standards

METRICS for design flow

Flow METRICS experiments

METRICS integration with datamining

Datamining integration experiments

Issues and conclusions

Flow MetricsFlow Metrics

Tool metrics alone are not enough Design process consists of more than one tool A given tool can be run multiple times Design quality depends on the design flow and

methodology (the order of the tools and the iteration within the flow)

Flow definition Directed graph G (V,E)

V T { S, F } T { T1, T2, T3, …, Tn } (a set of tasks) S starting node, F ending node E { Es1, E11, E12, …, Exy } (a set of edges)

Exy x < y forward path x = y self-loop x > y backward path

Flow ExampleFlow Example

S

T1

T2

T3

T4

F

Optional task

Task sequence: T1, T2, T1, T2, T3, T3, T3, T4, T2, T1, T2, T4

S

T1

T2

F

T1

T2

T3 T3 T3

T4

T2

T1

T2

T4

Flow TrackingFlow Tracking

Run Current FLOW_SEQUENCENo Task T1 T2 T3 T41 T1 1 - - - 12 T2 1 1 - - 1/13 T1 2 - - - 24 T2 2 1 - - 2/15 T3 2 1 1 - 2/1/16 T3 2 1 2 - 2/1/27 T3 2 1 3 - 2/1/38 T4 2 1 3 1 2/1/3/19 T2 2 2 - - 2/210 T1 3 - - - 311 T2 3 1 - - 3/112 T4 3 1 - 1 3/1/0/1

TASK_NOS

T1

T2

F

T1

T2

T3 T3 T3

T4

T2

T1

T2

T4

Task sequence: T1, T2, T1, T2, T3, T3, T3, T4, T2, T1, T2, T4

NELSIS Flow Manager IntegrationNELSIS Flow Manager Integration

Flow managed by NELSIS

Optimization of Incremental Multilevel FM PartitioningOptimization of Incremental Multilevel FM Partitioning Motivation: Incremental Netlist Partitioning

netlist ECOs are made; want top-down placement to remain similar to previous result

good approach [CaldwellKM00]: “V-cycling” based multilevel Fiduccia-Mattheyses

what is the best tuning of the approach for a given instance?

break up the ECO perturbation into multiple smaller perturbations?

#starts of the partitioner? within a specified CPU budget?

Optimization of Incremental Multilevel FM Partitioning (contd.)Optimization of Incremental Multilevel FM Partitioning (contd.) Given: initial partitioning solution, CPU budget and

instance perturbations (I)

Find: number of parts of incremental partitioning and number of starts

Ti = incremental multilevel FM partitioning Self-loop multistart n number of breakups (I = 1 + 2 + 3 + ... + n)

S T1 FT2 T3 Tn...

Multilevel FM Experiment Flow SetupMultilevel FM Experiment Flow Setup

foreach testcase

foreach I

foreach CPUbudget

foreach breakup

Icurrent = Iinitial

Scurrent = Sinitial

for i = 1 to n

Inext = Icurrent + i

run incremental multilevel FM partitioner

on Inext to produce Snext

if CPUcurrent > CPUbudget then break

Icurrent = Inext

Scurrent = Snext

end

Flow Optimization ResultsFlow Optimization Results

If (27401 < num edges 34826) and (143.09 < cpu time 165.28) and (perturbation delta 0.1) then num_inc_parts = 4 and num_starts = 3

If (27401 < num edges 34826) and (85.27 < cpu time 143.09) and (perturbation delta 0.1) then num_inc_parts = 2 and num_starts = 1

...

Actual CPU Time (secs)

Pre

dict

ed C

PU

Tim

e (s

ecs)

DesignName Optimized Regularibm01 215 217ibm05 1685 1723ibm02 249 269ibm03 618 669ibm06 363 371ibm04 444 488ibm08 1127 1219ibm10 752 773

Num Net Cut

Identifying the Effect of Wire Load ModelIdentifying the Effect of Wire Load Model Wire load model (WLM) is used for pre-layout

estimation of wire delays

Three different WLMs statistical WLM structural WLM custom WLM

Motivation: identify if WLMs are useful for estimation identify if WLMs are necessary for optimization identify the best role of WLMs

Wire Load Model FlowWire Load Model Flow

WLM flows for finding the appropriate role of WLM T1 = synthesis & technology mapping T2 = load wire load model (WLM) T3 = pre-placement optimization T4 = placement T5 = post-placement optimization T6 = global routing T7 = final routing T8 = custom WLM generation

ST1

T2

T3

T4

F

T5

T7

T8

T6

WLM Experiment SetupWLM Experiment Setup

foreach testcase

foreach WLM (statistical, structural, custom, and no WLM)

foreach flow variant

run PKS flow

if WLM = structural then

generate custom WLM

end Flow Sequence WLMF1 T1,T2,T3,T4,T5,T6,T7 StructuralF2 T1,T2,T3,T4,T5,T6,T7 StatisticalF3 T1,T2,T3,T4,T5,T6,T7 Custom*F4 T1,T4,T5,T6,T7 N/AF5 T1,T2,T3,T4,T5**,T6,T7 StatisticalF6 T1,T2,T3,T4,T6,T7 Statistical

* Custom WLMs are generated using flow F1 with additional task T8

** Special T5 run without logic restructuring

6 different flow variants

WLM Flow ResultsWLM Flow Results

Design Flow CPU SlackF1 210.00 -0.104F2 335.77 -0.280F3 157.29 -0.158

Design04 F4 649.05 -0.280F5 245.85 -0.719F6 53.17 -11.599F1 930.12 -0.430F2 780.77 -0.479F3 384.60 -0.396

Design08 F4 1442.58 -0.578F5 751.37 -0.441F6 287.69 -3.027F1 560.91 -0.760F2 746.05 -0.230F3 409.98 -1.000

Design12 F4 615.70 -0.640F5 834.05 -0.740F6 189.46 -14.310

Slack comparison for 6 flow variants

Post-placement and pre-placement optimizations are important steps

Choice of WLM depends on the design

WLMs are still useful

OutlineOutline

METRICS system architecture and standards

METRICS for design flow

Flow METRICS experiments

METRICS integration with datamining

Datamining integration experiments

Issues and conclusions

Datamining IntegrationDatamining Integration

DatabaseDatamining

Tool(s)

Datamining

Interface

Java

Servlet

Java

Servlet

SQL

SQL

Tables

Tables Tables

Results

ResultsDM

Requests

Inter-/Intranet

Categories of Data for DataMiningCategories of Data for DataMining

Design instances and design parameters attributes and metrics of the design instances e.g., number of gates, target clock frequency, number

of metal layers, etc.

CAD tools and invocation options list of tools and user options that are available e.g., tool version, optimism level, timing driven option,

etc.

Design solutions and result qualities qualities of the solutions obtained from given tools and

design instances e.g., number of timing violations, total tool runtime,

layout area, etc.

Possible Usage of DataMiningPossible Usage of DataMining

Design instances and design parameters

CAD tools and invocation options

Design solutions and result qualities

Given and , estimate the expected quality of e.g., runtime predictions, wirelength estimations, etc.

Given and , find the appropriate setting of e.g., best value for a specific option, etc.

Given and , identify the subspace of that is “doable” for the tool e.g., category of designs that are suitable for the given

tools, etc.

Example Applications with DMExample Applications with DM

Parameter sensitivity analysis input parameters that have the most impact on results

Field of use analysis limits at which the tool will break tool sweet spots at which the tool will give best results

Process monitoring identify possible failure in the process (e.g., timing

constraints are too tight, row utilization is too high, etc.)

Resource monitoring analysis of resource demands (e.g., disk space,

memory, etc.)

DM Results: QPlace CPU TimeDM Results: QPlace CPU Time

If (num nets 7332) then CPU time = 21.9 + 0.0019 num cells + 0.0005 num nets + 0.07 num pads - 0.0002 num fixed cells

If (num overlap layers = 0) and (num cells 71413) and (TD routing option = false) then CPU time = -15.6 + 0.0888 num nets - 0.0559 num cells - 0.0015 num fixed cells - num routing layer

...

Actual CPU Time (secs)

Pre

dict

ed C

PU

Tim

e (s

ecs)

Selection for Training and Test SetsSelection for Training and Test Sets

Random Case randomly select runs assigned to training set leave all remained (unselected) runs for test set

Distinct Case split the test cases into two distinct sets, the training

set and the test set assign the runs accordingly

Representative Case split the test cases into two distinct sets and assign

the runs accordingly for each test case in the test set, move exactly one run

to the training set I.e., for each test case, there is at least one

representative run in the training set

Prediction Result VariancesPrediction Result Variances

Random Case Distinct Case

RepresentativeCase

Prediction ofQP Wirelength

CTGen ResultsCTGen Results

Max InsertionDelay (ns)

Min InsertionDelay (ns)

Max Skew

OutlineOutline

METRICS system architecture and standards

METRICS for design flow

Flow METRICS experiments

METRICS integration with datamining

Datamining integration experiments

Issues and conclusions

ConclusionsConclusions

Extensions to current METRICS system is presented

Complete prototype of METRICS system is working at UCLA with Oracle8i, Java Servlet and XML (other working prototypes are installed at Intel and Cadence)

METRICS wrappers for Cadence, Synopsys and UCLA tools and flows

METRICS system is integrated with Cubist datamining tool and NELSIS flow manager

A complete METRICS system can be installed on a laptop and configured to work behind firewalls

Issues and Ongoing WorkIssues and Ongoing Work

Issues for METRICS constituencies to solve security: proprietary and confidential information standardization: flow, terminology, data management,

etc. social: “big brother”, collection of social metrics, etc.

Ongoing work with EDA, designer communities to identify tool metrics of interest users: metrics needed for design process insight,

optimization vendors: implementation of the metrics requested,

with standardized naming / semantics

http://vlsicad.cs.ucla.edu/GSRC/METRICS