Post on 20-Jul-2020
transcript
March 3 - 6, 2019
Hilton Phoenix / Mesa Hotel Mesa, Arizona
Archive
COPYRIGHT NOTICEThe presentation(s)/poster(s) in this publication comprise the proceedings of the 2019 TestConX workshop. The content reflects the opinion of the authors and their respective companies. They are reproduced here as they were presented at the 2019 TestConX workshop. This version of the presentation or poster may differ from the version that was distributed in hardcopy & softcopy form at the 2019 TestConXworkshop. The inclusion of the presentations/posters in this publication does not constitute an endorsement by TestConX or the workshop’s sponsors.
There is NO copyright protection claimed on the presentation/poster content by TestConX. However, each presentation/poster is the work of the authors and their respective companies: as such, it is strongly encouraged that any use reflect proper acknowledgement to the appropriate source. Any questions regarding the use of any materials presented should be directed to the author(s) or their companies.
“TestConX” and the TestConX logo are trademarks of TestConX. All rights reserved.
www.testconx.org
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
TestConX 2019
Michael Campbell
Sr. Vice President of Engineering
Qualcomm Technologies Inc. (QTI)
March 2019
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality.
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
22
“The world’s most valuable resource is no longer oil, but data”May 6th, 2017 The Economist.
Data is integral to optimizing semiconductorsTest drives immense amount of dataFor QTI and many others, data can be measured in terabytes per month
Yield
Burn in Time to Market
Quality I P Management
Test Time
Actionable data
mjc -testConex
Semiconductor data sources
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
333
A treasure chest of data, but how you optimize that much data ? Optimized databases and machine learning are some of the keys.
• QTI Semiconductor Landscape
• Major Technology Node or foundry changes every 1 – 2yrs
• 5x more data to analyze on every new technology node (sensors, transistors, conditions, process, etc)
• Requirements:
• Faster time to yield
• Shorter time to root cause yield loss
• Test, Process or Design
• Traditional methodologies do not work well due to complexity and time constraints
• To many transistors
• To many interactive variables
• To much data
Source: Qualcomm Technologies datamjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
44Source: Qualcomm Technologies data
The amount of transistors has increased over 5x as die size decreased over 50%
In our products, additionally the standard cell and memory bits have increased ~ 3-4x
Technology Migration has Enabled Qualcomm® SnapdragonTM Mobile Platform Capabilities ……..
Source: Qualcomm Technologies data
Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc. and/or its subsidiaries
but with some challenges.
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
55
RF transceiver complexity over time
• Data rates are up year over year - more Uplink + Downlink needed to keep up with data rate needs• Peak Data rates increased ~8x from 2015 thru 2019 while RF complexity increased >1000X.• Result: 5G RF will more 100-1000 x data taken during characterization and potentially manufacturing
4G Gen1
4G Gen2
4G Gen3
5G Gen1
RF Device Complexity 264 12096 207360 1373760Max Data Rate (Gbps) 0.6 1 2 5
0
1
2
3
4
5
6
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
1,600,000
Max
Dat
a Ra
te (G
bps)
RF C
ompl
exity
(Car
rier
Aggr
egat
ion
case
s)
DL, UL capability`
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
66Source: Qualcomm Technologies data
To meet customer demands requires faster time to yield and qualifty !!
Source: Qualcomm Technologies data
Industry demands are driving more capability and faster time to commercialization.
Source: Qualcomm Technologies data
0
30
60
90
120
0
5
10
15
20
25
30
201028nm
2011 2012 201328nm
HP
2014 201520nm
201614nm
201710nm
2018 20197nm
Mas
k Le
vels
Mon
ths
QTI Design to Volume ShipsTime to CS from ES
Time from CS to 1 million units
Time from CS to 10 million units
Mask Levels
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
7
QTI manufacturing process data collection ecosystem
Wafer WAT
Die Finished Good
Assembly TestWafer SortBumpFoundry
In Line Bump Process
B2B File Processing
BumpBE2/
Assembly
Final Test/QA
Wafer Sort SLTWAT
HadoopData validation, file processing and storage
these demandsWorking to build data solutions to meet these demands
When does data get sent to QTI?
SLT
2nd level assemblyl
FP Module
Machine Learning & Advanced Analytics
2nd level assembly
System in a Package
OptimalYield Explorer
Design-centric yield management
PDF Solutions
FT SLT
Fab
Source: Qualcomm Technologies data
Smartest 8Enabling more for
advanced analytics for
digital
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
88
- 200 400 600 800
1,000 1,200 1,400 1,600 1,800 2,000 2,200
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018est
2019est
2020est
2021est
Manufacturing data continues to increase
Increased design/process/test/In Line data is driving data volume.Machine learning increasingly required to detect data shifts.
45nm 28nm
Com
pres
sed
TB
Calendar Years
Cumulative data volume over time
7nm10nm14nm20nm
4x growth
10x growth
Source: Qualcomm Technologies data
Source: Qualcomm Technologies data
Source: Qualcomm Technologies datamjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
9
Inbound AP
I / Loaders
ClusterSQL Engine &
Analytics Library
Application Interface
Distributed MPPDatabase
Recent Summary & Detailed Data
Meta-data
Distributed File SystemHadoop
All Data Query-able FormatA
rchi
ve
Big Data Architecture
Structured Database2
1
3
Inbo
und
Loa
der
Pro
cess
Application Interface
Structured Database
Online Summary & Detailed DataSQL Engine & Application
Server
Client-Server Architecture
Data volume is forcing “Big Data” architectures on analytic applicationsLegacy architectures can’t keep up
Impacts• Data volumes are too large
for a traditional relational database to manage
• Data loading, database & client performance degrade
• Database becomes unmanageable
• Scaling is complex & costly
Results• Migration to big data becomes required to maintain usability
• Increasing • Files sizes.. more tests, more parameters, more die per wafer• Data types.. increased complexity of designs & processes, modules• Analysis.. larger retrievals across operations; cross data correlations
Changes1. Only key meta-data is
stored in structure database
2. Distributed MPP database improves performance & scalability for data analytics
3. Hadoop based batch processing used for larger datasets & timespans
Distributed File SystemHadoop
Raw Data Archive+ ETL to Application
4months of active data ~ 80-100 terabytes of data optimized ~10% meta data; Vertica (distributed db) has ~90% of the data
The new solution – shows 2x faster load , with query performance ~6x - 10x improvement especially on the unit/device level data
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
Machine Learning
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
11Reference from Optimal Plus.( O+)
Adapt
Learn Act
Validate
Use historical data to build and test a model
If model performance degrades => Refresh the model
Deploy the model and use it to do/decide/alert on something
Measure model performance as things change (all the time)
The Machine Learning Process
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
12
Tableau
Optimal
Operational Zone Descriptive Reporting
Diagnostic Analytics
Predictive Analytics
Graph Analytics
Machine Learning
Analytics Applications UsersSources
Inline
PCM / WATWafer map
KrigingWS1
WS2
RMA
FABData
BKM
Yield LossDrifting Params
Performance
CustomerData
YieldData
Assembly
FT1FT2
SLTFT3
QA
SATData
ExternalPartners
Vignesh
DataScientists
Land
ing
Stan
dard
izatio
n
Tran
sfor
mat
ions
&Fe
atur
e En
gine
erin
g
Integration
Ingest Prepare Consume
Exploration Zone for Data
Metadata Operational BusinessTechnical
Data Lake Data Warehouse Data Automation
Multiple products
…
Users
AnalyticData Sets TCE PTESD PTE QTI
Cost Opt.
TTR Burn-In
Process
DPPM
Data Models
Similar Business Cases
Common Summaries
Derived Cases
User Data
Best Performing
solution
Qualcomm Analytic Phase 1 statusQualcomm Analytic Phase 1 status
ForecastYield
Performance
SLT
Yield TeamRMA Team
InternalPartners
Version 0.13/20/2018
Self-Service Analytic Tools
Operational Apps
Data Visualization
PythonR
Analytic Applications
QTI Product and Test : Machine Learning System Overview
It’s all about the infrastructure:Need to have the relevant building blocks to create, validate, deploy and monitor ML models in production.The ML code itself is only a small part of the picture.
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
1313
Capacity limitation
Vertical Ramps
More complicateddesigns in smaller area
DPPM to BPPM
Time to market
Yield
Cost
Quality scale
Production EnvironmentNPI Challenges
$$$
Machine learning can be used to maximize the value of data to deliver yield, quality, and cost.
Source: Qualcomm Technologies data
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
14
Continuous challenges…
Balancing Quality and Cost of Test:
• Adapt fast to higher production demands w/o quality risk increases.
• Continuous optimization of throughput capacity across family tiers.
• Live KPI monitor to guarantee all levels are met.
• Continue DFT and Test innovation solutions to address new challenges proactively.
New technology nodes require higher number of tests:
• The 0.5% baseline fault coverage gap became TOO BIG to skip.
• EDA tools can’t cover ALL faults in production.
• New fault models are needed• ATE Vector Memory is limited• Higher test times constrain volume
More functional tests @ NPI :
• Covering inter-block defects• Provides gap feedback to DFT• Helps close gap with DPPM• Required with performance tests
Production Ramps are going “Vertical”
• From 9 months to 3 months in latest projects.
• No room for errors• TTR and monitor techniques in place
from day 1.• Identify effective tests ASAP
Smarter and more adaptive test programs are needed!!
Production ramps
# of ATE
050100150200250300350400450500
0100020003000400050006000700080009000
10000
2008 2010 2012 2014 2016 2018 2020
0.5%
Unc
over
ed T
rans
isto
r Gap
(Mill
ions
)
Tran
sist
or C
ount
(Mill
ions
)
Year
Transistor Count vs 0.5% Uncovered Fault Transistor Gap
Source: Qualcomm Technologies data
Mobile industry is now driving toward “almost-automotive” quality levels (<< 100 DPPM)With nearly vertical ramps in new technologies.
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
151515
A treasure chest of data, using R and python( linked to O+) can create value add data.
People drive decision based on data, ML (Machine Learning) brings value add data!
• QTI semiconductor landscape Using machine learning
• Major Technology Node or foundry changes every 1 – 2yrs
• 5x more data to analyze on every new technology node (sensors, transistors, conditions, process, etc)
• Requirements:
• Faster time to yield
• Shorter time to root cause yield loss
• Test, Process or Design
• Benefits:
• Transforms high dimensionality problems into a simplified version for human analysis
• Self-train / adapts according to the fabrication & test conditions
• Detects systematic patterns related to areas of interest (yield loss, marginalities, HW, etc)
Source: Qualcomm Technologies datamjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
16
Machine learning examples Test time and Quality
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
17
Critical test List (aka: DO NOT
REMOVE)Quantify Escape Rates & TTRQuantify Escape Rates & TTR
Production:10% Sampling of “Lean” Devices* Used to monitor performance
Production:10% Sampling of “Lean” Devices* Used to monitor performance
Wafer Data Lots
OptimalOptimal
RRAnalytics / ML Algorithms
Identify:•Redundant test•0-DPPM test
Optimize:•Keeps DPPM < 30•Select effective @ defect screen with lowest TT
“LEAN” Test List
Add tests from escapees into “Critical List”
< 30 DPPM?
Continuous check of effective coverage with self-adaptation
NO
YES
XGB Model
PythonPython“LEAN” Test Prog (xx % TTR)
Full Test Program
PerformanceFalse Pass Prediction = 0.10 - 0.39% (Escapes)True Fails Predicted = >75%
Lean coverage with machine learning-Test Optimization and predicting Quality
TTR Savings up to 20% @ < 12 Escapes / mu
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
18
Forward Looking implementation in production
FabAdaptive
Probe Test
Bump Site
Fabrication Wafer Level
EngineeringData Base
PCM: Parametric Data
Probe: Yield & Parametric
QualModelQual
ModelFab ModelFab Model
AssyModelAssyModelA
ugm
ente
d Te
stsModel
PerformanceMonitoring
Adjust/Re-train
Meets criteria
Fails criteria
FFWD (Feed-Forward)
Advanced Analytics / ML models
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
19
Quality - Selective devices for Burn-In
Wafer Sort
FT @Temps
Wafer Stage Package Test
Production Parameters
NN
ML
Mod
els
DFFW Risk Class Index
RE-Train criteria from O+
Burn-In
RandomForest
RegressionLogistic
Regression
WAT
WS
FT Hot
FT Cold
FT Room
High Risk? Burn-In
Bypass
To Customer
Goal: Select ONLY the 50% highest failure probability devices for Burn-In stage test while keeping the same DPPM as existing process on record.
yes
no
Source: Qualcomm Technologies data
Read Point
Sample Check
Passing
Passing
Feedback to model retrain
Burn-In Stage
GDBN
DPAT
Samples of Bypass will go into Burn-In to validate passing predictions
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
20
DFT and fault models
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
214/1/2019
Process Defectivity
Time
NPIProduction
UntestableFaults
UntestableFaults
DFT test coverage gaps: Un-Scan-able (mixed signal, PLL, ..) and coverage lost due to test mode constraints & exceptions
NPIProduction
Immature process with high defectivity
Foreign Material
Non-fill
Line Break
Defect
Design /process margin sensitivities Micro defects leading to model to silicon sensitivities. Circuit sensitivities esp sensitive to process – VT shift, N/P skew
ATE test cost to cover these DFT holes & process defectivity is increased significant esp for Digital part > 5B transistors or ~500M faults
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
22
Testable Gates
Full SOC Logic (including Digital, Analog, MSIP)
~0.1%Logic present in full chip SOC that is not modelled for DFT tools. This could originate from custom logic, analog & mixed signal IPs and certain simplified ATPG models for Qualcomm Proprietary logic gates.
~0.50%A measurable portion of the design that is modelled for the DFT/ATPG tools is deemed untestable due to missing scan structures or test mode constraints, or asynchronous logic, that can prevent ATPG tools to test functional paths.
Logic Modelled for DFT
Logic Testable by SCAN
Faults for ATPG
Basic models: SAF, TDF
99%+detected
faults
Advanced: Cell Aware & others
90%+detected
faults
Aborted Faults
Aborted Faults
DFT & EDA challenges to meet < 100 DPPM requirements
Requires H/W ChangesSource Qualcomm data
mjc -testConex
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
23
DFT & EDA challenges to meet < 100 DPPM requirements
Test Execution Time
Source Qualcomm data
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
24
USB on ATE – on chip system test – cost and quality• ATE testing for functional and Scan tests have parallelism constraints due to pins and power for complex
SOC’s. Incorporating a HV SLT could improve throughput and cost due to massive parallelism if enough ATE content can be moved. Test cost could reduce ATE costs while eliminating perceived DFT limitations on ATE due cost reasons. ATPG on SLT could optimize test-cost.
Hypothetical test time breakdown
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
2525
Summary
Change – it’s a good thing.
• Excel is still a powerful tool, however,
• Machine learning leverages excel, while maximizing the data reviewed and minimizes the time to value add data by > 100X.
• Machine learning DOEs have provided value add in TT, yield, and quality analysis.
• Test over USB is a way to potentially lower COT and overall cost.
• High volume SLT’s may be in the cards for the future
• Automating the mundane machine learning could unleash more engineering creativity
• Lets look at one view of the future. From ITC 2018, author DR Li-C. Wang, and IEA project https://www.youtube.com/watch?v=HpajqoRdz-Q
Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote
March 3-6, 2019TestConX Workshop www.testconx.org
Nothing in these materials is an offer to sell any of the components or devices referenced herein.
©2019 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
Qualcomm, Snapdragon, and MSM are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Other products and brand names may be trademarks or registered trademarks of their respective owners.
References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.
Follow us on:For more information, visit us at:www.qualcomm.com & www.qualcomm.com/blog
Thank you