Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron Microscope
C.F. Batten, D.M. Holburn, B.C. Breton, N.H.M. CaldwellScientific Imaging Group
Cambridge University
ScanningMay 7th, 2001
May 7th, 2001 Scanning 2
Motivation
• There is a growing need for instrument automation– New applications have led to an increase in the
number of novice SEM operators– Remote microscopy requires simple commands
which perform more work
• Focusing is an ideal candidate for automation– Simplifies a common and tedious operation– Helps make remote microscopy practical
May 7th, 2001 Scanning 3
Previous Work
• Scanning Electron Microscopy– Software solution using image gradient [Tee79]– Hardware solution using image covariance [Erasmus82] – Software solution using frequency domain [Ong98, Ogasawara99]– Use of a general imaging model to predict best focus [Nicolls95]
• Optical Microscopy– Survey of sharpness measures [Groen85, Firestone91]– Use of a Fibonacci search to find the best focus [Yeo93]
May 7th, 2001 Scanning 4
Our Approach
• Traditional autofocusing approaches – Try to integrate additional functionality such as astigmatism
correction or topological mapping– Use a fixed stepsize or iterative search and avoid more
sophisticated search algorithms due to low SNR and hysteresis concerns
• Our approach– Make a dedicated autofocusing search algorithm
Increase the SNR for
each image
Use a more sophisticated
search algorithm
Decrease the number of required
image captures
May 7th, 2001 Scanning 5
Outline
• Sharpness Measures– Gradient measure– Frequency domain measure– Autocorrelation measures– Variance measure
• Sharpness Search Algorithms– Fixed stepsize search– Fixed stepsize search with interpolation– Iterative search– Variable stepsize search– Fibonacci search
• Conclusions
May 7th, 2001 Scanning 6
Evaluating Sharpness Measures
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
10
20
30
40
50
60
Focus Step (0.01mm)
Var
ianc
e S
harp
ness
Mea
sure
Forward Sweep 1Forward Sweep 2Forward Sweep 3
Strictly Unimodal Property
Sharpness measure should have one peak at the best focus and strictly decrease away from this maximum
May 7th, 2001 Scanning 7
Gradient Measure
• Sum of the difference between every nth pixel in both the X and Y directions
• As image comes into focus, edges become sharper increasing the image gradient
• Sharpness Measure Properties– Relatively easy to calculate (one of the first sharpness measures)– Very susceptible to noise – The parameter n acts a low-pass filter in the spatial domain
• (n = 1) Traditional image gradient• (n = 2) Brenner method• (n > 2) As long as n < feature size, can increase noise robustness
May 7th, 2001 Scanning 8
Frequency Domain Measure
• Perform Fourier transform and then sum the frequency components below threshold frequency (Ω)
• As image comes into focus, edges become sharper which increases the magnitude of medium frequency components
• Sharpness Measure Properties– Allows easy integration of astigmatism correction– Fourier transform in software is computationally expensive– The parameter Ω acts as a low-pass filter in frequency domain
• Varying Ω produces similar results as varying n• For this work, Ω chosen to be 50
May 7th, 2001 Scanning 9
Auto-correlation Measures
• Auto-correlation function is the image convolved with itself and indicates how well neighboring pixels are correlated
• Tested two measures using the image auto-correlation – ACFdiff Height of the central ACF peak– ACFsum Area under the central ACF peak
• Focused images contain small highly correlated regions that result in a tall sharp central ACF peak
• Sharpness Measure Properties– Can calculate ACF efficiently in the frequency domain– Do not need to calculate entire ACF for ACFdiff measure– Correlated noise due to limited bandwidth distortion made
using the ACF more difficult
May 7th, 2001 Scanning 10
Variance Measure
• Sum the square of the difference between each pixel and the mean image intensity
• Focused images have greater intensity variation then blurred defocused images
• Sharpness Measure Properties– Simple and efficient implementation– Very robust to noise– Strong adherence to the strict unimodality property
For these reasons the variance measure was selected as the primary sharpness measure for this work
May 7th, 2001 Scanning 11
Comparison of Sharpness Measures
0 10 20 300.4
0.6
0.8
1
(a) fgrad
0 10 20 30−0.5
0
0.5
1
(c) fACFdiff
0 10 20 30
0.2
0.4
0.6
0.8
1
(b) ffreq
0 10 20 300
0.2
0.4
0.6
0.8
1
(e) fvar
Pix Avg: 1 FAvg: 3 Pix Avg: 1 FAvg: 5 Pix Avg: 2 FAvg: 3 Pix Avg: 2 FAvg: 10 Pix Avg: 4 FAvg: 15 Pix Avg: 8 FAvg: 20 Pix Avg: 16 FAvg: 25
0 10 20 300
0.2
0.4
0.6
0.8
1
(d) fACFsum
Sharpness Measures at Various Noise Levels
May 7th, 2001 Scanning 12
Sharpness Search Algorithms
• Investigated five sharpness search algorithms– Fixed stepsize search– Fixed stepsize search with interpolation– Iterative search– Variable stepsize search– Fibonacci search
• Notation– l Search interval– α Desired accuracy (How close to optimum is acceptable?)– N Number of required image captures
• Goal is to find a search algorithm which minimizes Nbut still achieves the desired accuracy α
May 7th, 2001 Scanning 13
Fixed Stepsize Search
• Single sweep over search interval with stepsize = 2α• Theoretical N given by
• Peak finding reduces N• Developed a novel method
to adjust for hysteresis effects based on relative sharpness when returning to best focus
4 4.5 5 5.5 6 6.5400
500
600
700
800
900
1000
1100
Focal Length (mm)
Var
ianc
e S
harp
ness
Mea
sure
Variance Moving Average
+= 12αlN
Fixed Stepsize Search with Peak Finding
May 7th, 2001 Scanning 14
Fixed Stepsize with Interpolation
• Interpolation can help reduce the number of image captures while maintaining the desired accuracy
• Quadratic and linear interpolation do not perform well on typical variance curves
• A New Interpolation Approach– Erasmus and Smith provide a derivation for image variance
as a function of defocus [Erasmus82]– Use non-linear regression to curve fit the derived function
with the collected data– This allows us to significantly reduce the required number of
image captures, but is computationally expensive
May 7th, 2001 Scanning 15
Fixed Stepsize with Interpolation
8.5 8.61 8.72 8.83 8.940.2
0.4
0.6
0.8
1
1.2
1.4
Focal Length (mm)
Nor
mal
ized
Var
ianc
e
(a) Gold on Carbon − 22,200x
8.56 8.6 8.65 8.69 8.740.2
0.4
0.6
0.8
1
1.2
Focal Length (mm)
Nor
mal
ized
Var
ianc
e
(b) Gold on Carbon − 75,500x
8 8.66 9.33 10 10.66
0.6
0.8
1
1.2
Focal Length (mm)
Nor
mal
ized
Var
ianc
e
(c) IC Tracks − 3,400x
6 7.55 9.11 10.66 12.220.6
0.7
0.8
0.9
1
1.1
1.2
Focal Length (mm)
Nor
mal
ized
Var
ianc
e
(d) IC Tracks − 3,400x
8.69 8.98 9.27 9.56 9.850.5
0.6
0.7
0.8
0.9
1
1.1
Focal Length (mm)
Nor
mal
ized
Var
ianc
e
(e) IC Tracks − 9,900x
7 8.33 9.66 11 12.330.4
0.6
0.8
1
1.2
1.4
Focal Length (mm)N
orm
aliz
ed V
aria
nce
(f) Sublimated Titanium − 1,500x
Derived Variance Function Fitted to Data from Various Specimens
May 7th, 2001 Scanning 16
Iterative Search
• Several sweeps with gradually smaller stepsizes and search intervals
• Theoretical N given by
where η is the number of image captures per iteration
=
1))-η/(2log()/αlog(η lN
6 6.5 7 7.5 8 8.5 9 9.5300
350
400
450
500
550
600
Focal Length (mm)
Var
ianc
e S
harp
ness
Mea
sure
Iteration 1Iteration 2Iteration 3Iteration 4
Online Iterative Focus Sweep (η=8)
May 7th, 2001 Scanning 17
Variable Stepsize Search
• Reduce the stepsize as the sharpness increases
• A common technique in other maximum search problems, but not used in SEM autofocusing due to low image SNR
• We adapt the algorithm as follows– Reduce stepsize based on moving average of variance– Set 2α as a lower bound on the stepsize – Use peak finding– Perform final fixed stepsize search if stepsize
is greater than 2α once the peak is found
• Actual number of image captures varies based on initial stepsize and specific variance curve
May 7th, 2001 Scanning 18
Variable Stepsize Search
4 4.5 5 5.5 6 6.5400
500
600
700
800
900
1000
1100(a) Sharpness During Variable Stepsize Sweep
Focal Length (mm)
Var
ianc
e S
harp
ness
Mea
sure Variance
Moving Average
4 4.5 5 5.5 6 6.5
0.1
0.15
0.2
0.25
0.3
0.35(b) Stepsize During Variable Stepsize Search
Focal Length (mm)
Ste
psiz
e (m
m)
Example of Variable Stepsize Search
May 7th, 2001 Scanning 19
Fibonacci Search
• An iterative search where η = 1• Use previous measurements and one new measurement to
narrow search interval• To avoid adverse hysteresis
effects, must set instrument to small focal length before each image capture (~200ms)
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34400
500
600
700
800
900
1000
1100
Focal Length (arbitrary units)
Sha
rpne
ss M
easu
re
•
•
•
•
•
••
1
2
3
4
5
6
7
a b
Fibonacci Search Image Captures
May 7th, 2001 Scanning 20
Fibonacci Search
7.1 7.15 7.2 7.25 7.3 7.35 7.4200
400
600
800
1000
1200
1400
1600
1800
Focal Length (mm)
Var
ianc
e S
harp
ness
Mea
sure
Fixed Stepsize SearchFibonacci Search
Example of Fibonacci Search
May 7th, 2001 Scanning 21
Results: Number of Image Captures
0.010.020.030.040.050.060.070.080.090.10
10
20
30
40
50
60
70
80
90
100
Relative Accuracy (accuracy/search interval)
Num
ber
of R
equi
red
Imag
e C
aptu
res
Fixed Stepsize Search Iterative Search (η = 8) Variable Stepsize Search (Test #1)Variable Stepsize Search (Test #2)Fibonacci Search
Total Search Time vs. Relative Accuracy
May 7th, 2001 Scanning 22
Results: Total Search Time
0.0140.0160.0180.020.0220.0240.0260.0280.030.0320
20
40
60
80
100
120
140
160
Relative Accuracy (accuracy/search interval)
Tim
e (s
econ
ds)
Fixed Stepsize Search with Interpolation Iterative Search Variable Stepsize SearchFibonacci Search
Total Search Time vs. Relative Accuracy
May 7th, 2001 Scanning 23
Results: Relative Sharpness
• Hysteresis effects prevent us from just comparing the best focus produced by each sharpness search algorithm
• Use relative sharpness as a more accurate metric
(a) Gold on Carbon25,800x
(b) Integrated Circuit970x
(c) Sublimated Titanium1,350x
(d) Etched Silicon410x
Specimens Used for Relative Sharpness Tests
May 7th, 2001 Scanning 24
Results: Relative Sharpness
FIX ITP ITR VAR FIB440
460
480
500
520
540(b) Integrated Circuit
Bes
t Foc
us V
aria
nce
FIX ITP ITR VAR FIB1000
1100
1200
1300
1400(c) Sublimated Titanium
Bes
t Foc
us V
aria
nce
FIX ITP ITR VAR FIB380
400
420
440
460(d) Etched Silicon
Bes
t Foc
us V
aria
nce
FIX ITP ITR VAR FIB1400
1600
1800
2000
2200
2400
2600(a) Gold on Carbon
Bes
t Foc
us V
aria
nce
Best Focus Sharpness for Various Search Methods
May 7th, 2001 Scanning 25
Conclusions
1. The variance measure is an effective sharpness measure that is well suited for autofocusing in the scanning electron microscope.
2. Autofocusing research has traditionally concentrated on fixed stepsize and iterative searches, but more sophisticated search algorithms can successfully reduce the total search time.