PowerPoint File available: ~jamesh/powerpoint/ Oslo_2010.ppt.

Post on 16-Jan-2016

218 views 0 download

Tags:

transcript

PowerPoint File available:

http://bl831.als.lbl.gov/

~jamesh/powerpoint/

Oslo_2010.ppt

http://ucxray.berkeley.edu/~jamesh/elves

Download Elves from:

Advanced Light Source

Beamline 8.3.1 staff

Acknowledgments

George MeigsGeorge Meigs

Jane Tanamachi

ALS 8.3.1 Team

Acknowledgements

8.3.1 PRT: Jamie CateCenter for Structure of Membrane Proteins

Membrane Protein Expression Center IICenter for HIV Accessory and Regulatory Complexes

W. M. Keck FoundationPlexxikon, Inc.

M D Anderson CRCUniversity of California Berkeley

University of California San FranciscoNational Science Foundation

University of California Campus-Laboratory Collaboration GrantHenry Wheeler

The Advanced Light Source is supported by the Director, Office of Science, Office of Basic Energy Sciences, Materials Sciences Division, of the US Department of Energy under contract No. DE-AC02-05CH11231 at Lawrence Berkeley National Laboratory.

Ken Frankel Chris Neilson Michael Blum Joe Ferrara

Elves examine images andset-up data processing

Elves run…

mosflmscalasolve

mlpharedm

arp/warp

Elven Automation

Elven Automation

Elves examine images andset-up data processing

Elves run…

mosflmscalasolve

mlpharedm

arp/warp

Elven Automation

Elves examine images andset-up data processing

Elves run…

mosflmscalasolve

mlpharedm

arp/warp

Conversational User Interface

user input ->

Conversational User Interface

user input -> process the data in /data/semet

Conversational User Interface

user input -> process the data in /data/semet

recognition blah blah blah blah /data/semet

Conversational User Interface

user input -> process the data in /data/semet

recognition blah blah blah blah /data/semet

simplify /data/semet

Conversational User Interface

user input -> process the data in /data/semet

recognition blah blah blah blah /data/semet

simplify /data/semet

preempt /data/semet contains image files

Conversational User Interface

user input -> process the data in /data/semet

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays.

Conversational User Interface

user input -> process the data in /data/semet

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays.

confirm Everything look okay? [Yes] ->

Conversational User Interface

user input -> process the data in /data/semet

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays.

confirm Everything look okay? [Yes] ->user input distance was more like 110

Conversational User Interface

user input -> distance was more like 110

Conversational User Interface

user input -> distance was more like 110

recognition distance blah blah blah 110

Conversational User Interface

user input -> distance was more like 110

recognition distance blah blah blah 110

simplify DISTANCE 110

Conversational User Interface

user input -> distance was more like 110

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays.

Conversational User Interface

user input -> distance was more like 110

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays.

confirm Everything look okay? [Yes] ->

Conversational User Interface

user input -> distance was more like 110

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays.

confirm Everything look okay? [Yes]user input -> wavelength is wrong

Conversational User Interface

user input -> wavelength is wrong

Conversational User Interface

user input -> wavelength is wrong

recognition wavelength = -NOT

Conversational User Interface

user input -> wavelength is wrong

recognition wavelength = -NOT

simplify WAVELENGTH

Conversational User Interface

user input -> wavelength is wrong

recognition wavelength = -NOT

query What is the x-ray wavelength? [1.54Å] ->

simplify WAVELENGTH

Conversational User Interface

user input -> wavelength is wrong

recognition wavelength = -NOT

query What is the x-ray wavelength? [1.54Å]user input -> 1

simplify WAVELENGTH

Conversational User Interface

user input -> wavelength is wrong

query What is the x-ray wavelength? [1.54Å]user input -> 1

Conversational User Interface

user input -> wavelength is wrong

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays.

query What is the x-ray wavelength? [1.54Å]user input -> 1

Conversational User Interface

user input -> wavelength is wrong

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays.

query What is the x-ray wavelength? [1.54Å]user input -> 1

confirm Everything look okay? [Yes] ->

Conversational User Interface

user input -> wavelength is wrong

report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays.

query What is the x-ray wavelength? [1.54Å]user input -> 1

confirm Everything look okay? [Yes]user input -> Yes

Major Phasing techniques

• Molecular Replacement

• Multiple Isomorphous Replacement

• Multiwavelength Anomalous Diffraction

• Single-wavelength Anomalous Diffraction

?

Molecular Replacementcorrect structure and intensities

http://www.ysbl.york.ac.uk/~cowtan/fourier/coeff.html

Molecular Replacementuse something similar as a starting model

Model Buildingcurrent model is missing something

Model Buildingphases from model

Model Buildingmissing bits show up in “difference map”

Model Buildingmissing bits show up better in FO + (FO - FC) map

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

3 params

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

3 params

10 params

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

Major Phasing techniques

• Molecular Replacement

• Multiple Isomorphous Replacement

• Multiwavelength Anomalous Diffraction

• Single-wavelength Anomalous Diffraction

inverse Fourier Transformno phase

inverse Fourier Transformno phase

Major Phasing techniques

• Molecular Replacement

• Multiple Isomorphous Replacement

• Multiwavelength Anomalous Diffraction

• Single-wavelength Anomalous Diffraction

dete

ctor

anomalous scattering

sam

ple

x-ray beam

dete

ctoranomalous scattering

sam

ple

x-ray beam

Independent tasks can be performed simultaneously

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

Multiprocessing Strategy

SOLVE/P212121/doneSOLVE/P21212/doneSOLVE/P21221/doneSOLVE/P22121/doneSOLVE/P2221/doneSOLVE/P2212/doneSOLVE/P2122/busySOLVE/P222/done

Multiprocessing StrategyMultiprocessing Strategy

epmr/P212121/model1/doneepmr/P212121/model2/doneepmr/P21212/model1/doneepmr/P21212/model2/doneepmr/P21221/model1/doneepmr/P21221/model2/doneepmr/P22121/model1/busyepmr/P22121/model2/doneepmr/P2221/model1/epmr/P2221/model2/epmr/P2212/model1/epmr/P2212/model2/epmr/P2122/model1/epmr/P2122/model2/

Multiprocessing Strategy

wARP/P212121/donewARP/P21212/donewARP/P21221/donewARP/P22121/donewARP/P2221/donewARP/P2212/donewARP/P2122/busywARP/P222/done

Multiprocessing Strategy

space group

FOM Rcryst

P3221

P3121

P321

Multiprocessing Advantage

space group

FOM Rcryst

P3221 0.836 0.323

P3121

P321

Multiprocessing Advantage

space group

FOM Rcryst

P3221 0.836 0.323

P3121 0.865 0.196

P321 0.726 0.347

Multiprocessing Advantage

table1.com

table1.com

Elven Automation

How often does it really work?

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

148 datasets (15 MAD)

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

148 datasets (15 MAD)

31 investigators

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

148 datasets (15 MAD)

31 investigators

56 unique cells

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

148 datasets (15 MAD)

31 investigators

56 unique cells

5 KDa – 23 MDa asymmetric unit

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

148 datasets (15 MAD)

31 investigators

56 unique cells

5 KDa – 23 MDa asymmetric unit

0.94 – 32 Å resolution (3.2 Å)

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Number Description Percent

446028 Images (~7 TB) 33%

2346 Data sets 47%

449 MAD/SAD (1:2) 19%

104 Published 4.4%

8.3.1 in 2003

How many structures get solved?

Why do structures fail?

Overlaps

Why do structures fail?

Overlaps

Signal to noise

Why do structures fail?

Overlaps

Signal to noise

Radiation Damage

Why do structures fail?

Overlaps

Signal to noise

Radiation Damage

Why do structures fail?

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

unavoidable overlaps

unavoidable overlaps

dete

ctor

unavoidable overlaps

phi

dete

ctor

unavoidable overlaps

mosaicity

phi

dete

ctor

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

Ewald sphere

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

b

c

a

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

b

c

a

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

b

c

a

unavoidable overlaps

mosaicity

phi

dete

ctor

c*

b

c

a

avoidable overlaps

mosaicity

phi

dete

ctor

c*

b

ca

avoidable overlaps

mosaicity

phi

dete

ctor

c*

bc

a

avoiding overlaps

avoiding overlaps

avoiding overlaps

avoiding overlaps

c

c

avoiding overlaps

1000 mm

avoiding overlaps

1000 mm

2 mrad

10 seconds

avoiding overlaps

1000 mm

2 mm

2 mrad

10 seconds

avoiding overlaps

1000 mm

1 mm

1 mrad

20 seconds

avoiding overlaps

1000 mm

300 um

0.3 mrad

60 seconds

Overlaps

Signal to noise

Radiation Damage

Why do structures fail?

Overlaps

Signal to noise

Radiation Damage

Why do structures fail?

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Radiation Damage

why not just avoid it?

Holton & Frankel (2010) Acta D 66 393-408.

B ≈ 4 d2 + 120

20

4

0

60

8

0

100

12

0

aver

age

atom

ic B

fac

tor

1 1.5 2 2.5 3 3.5 4 4.5 5

resolution (Å)

Simulated diffraction imageSimulated diffraction imageMLFSOMMLFSOM

simulatedsimulated realreal

signal vs noise

signal vs noise

signal vs noise

“If you don’t have good data,

then you have no data at all.”

-Sung-Hou Kim

signal vs noiseeasy

hard

impossible

signal vs noiseeasy

hard

impossible

threshold of “solvability”

signal vs noise

“If you don’t have good data,

then you must learn statistics.”

-James Holton

Adding noise

Adding noise

12 + 12 = 1.42

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

σtotal2 = σ1

2 + σ22

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

σtotal2 = σ1

2 + σ22

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

σtotal2 = σ1

2 + σ22

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

102 + 12 = 10.052

MAD phasing simulation

-0.2

0

0.2

0.4

0.6

0.8

1

0.01 0.1 1 10

Anomalous signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o co

rrec

t m

odel

mlphare results

SAD phasing simulation

-0.2

0

0.2

0.4

0.6

0.8

1

0.01 0.1 1 10

Anomalous signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o co

rrec

t m

odel

mlphare results

SAD phasing experiment

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.5 1.0 1.5 2.0 2.5 3.0

Anomalous signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o pu

blis

hed

mod

el

MR simulation

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.001 0.01 0.1 1 10 100 1000

Signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o co

rrec

t de

nsity

corrupted data

“photon counting”

Read-out noise

Shutter jitter

Beam flicker

spot shape

radiation damage

σ(N) = sqrt(N)

rms 11.5 e-/pixel

rms 0.57 ms

0.15 %/√Hz

pixels? mosaicity?

B/Gray?

signal vs noise

Which error dominates?

• Weak spots (high-res)background

• MAD/SAD (small differences)detector calibration

( if not rad dam! )

Holton & Frankel (2010) Acta D 66 393-408.

Holton & Frankel (2010) Acta D 66 393-408.

Background level sets needed photons/spot

Moukhametzianov et al. (2008). Acta Cryst. D 64, 158-166

Holton & Frankel (2010) Acta D 66 393-408.

Optimal exposure time(faint spots)

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

too long!

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

Optimal exposure time(faint spots)

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

too short!

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

Optimal exposure time(faint spots)

σtotal2 = Nphotons

+ σreadout2+ σraddam

2

needlessly long

Nphotons ≈ σdetector2

Optimal exposure time(faint spots)

“optimal”

Nphotons ≈ 10x σdetector2

Optimal exposure time(faint spots)

“buried”

Nspot + Nbg ≈ 10x m

Optimal exposure time(faint spots)

“buried”

2

0

gain

0 + Nbg ≈ 10x m

Optimal exposure time(faint spots)

“buried”

2

0

gain

Optimal exposure time(faint spots)

2

00 10

gain

mgain

bgbghr

Optimal exposure time(faint spots)

2

00 10

gain

mt

t

gain

bgbg

ref

hrref

Optimal exposure time(faint spots)

0

2010

bgbggain

mtt

refrefhr

thr Optimal exposure time for data set (s)tref exposure time of reference image (s)bgref background level near weak spots on

reference image (ADU)bg0 ADC offset of detector (ADU)bghr optimal background level (via thr)σ0 rms read-out noise (ADU)gain ADU/photonm multiplicity of data set (including partials)

Short answer:

bghr ~ 100 ADU

for ADSC Q315r

What error dominates?

• Weak spots (high-res)background

• MAD/SAD (small differences)detector calibration

if not rad dam!

Optimal exposure time(anomalous differences)

I-I+

3%

100

phot

ons

10 photons

100

phot

ons

Optimal exposure time(anomalous differences)

I-I+

3%

100

phot

ons

14 photons

100

phot

ons

Optimal exposure time(anomalous differences)

3%

I-I+

2000

pho

tons

67 photons

Optimal exposure time(anomalous differences)

1%

I-I+

20,0

00 p

hoto

ns

200 photons

Minimum required signal (MAD/SAD)

"#

)(3.1

fsites

DaMW

sd

I

Holton & Frankel (2010) Acta D 66 393-408.

“photon counting”

Read-out noise

Shutter jitter

Beam flicker

spot shape

radiation damage

σ(N) = sqrt(N)

rms 11.5 e-/pixel

rms 0.57 ms

0.15 %/√Hz

pixels? mosaicity?

B/Gray?

signal vs noise

Optimal exposure time(anomalous differences)

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

Optimal exposure time(anomalous differences)

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

Optimal exposure time(anomalous differences)

0MADt

no detector is perfectly calibrated!

σtotal2 = Nspot + σbg

2 + σreadout

2+ σraddam2

+ (fshutterNspot )2 + (fflickerNspot )2

+ (fcalibNspot )2

σtotal2 = Nspot + σbg

2 + σreadout

2+ σraddam2

Fractional error

mult > (—)2Rmerge

<ΔF/F>

Holton & Frankel (2010) Acta D 66 393-408.

Damage model system

67 consecutive data sets

67 consecutive data sets

Data quality vs exposure

0.5

0.55

0.6

0.65

0.7

0 20 40 60 80 100

model vs warp

Exposure time (min)

Cor

rela

tion

coef

ficie

nt

Data quality vs exposure

579

1113151719212325

0 20 40 60 80 100

3A bin

Exposure time (min)

Data quality vs exposure

15

17

19

21

23

25

27

29

0 20 40 60 80 100

overall

Exposure time (min)

Data quality vs exposure1.7

1.9

2.1

2.3

2.5

2.7

0 20 40 60 80 100

I/sig(I) < 2

Exposure time (min)

Res

olut

ion

limit

Data quality vs exposure0

0.020.040.060.080.1

0.120.140.160.180.2

0 20 40 60 80 100

3A bin

Exposure time (min)

Rsy

m

Experimentally-phased map

Experimentally-phased map

Damage changes absorbance spectrum

0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

Photon energy (eV)

coun

ts

Damage changes absorbance spectrum

0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

Photon energy (eV)

coun

ts

Damage changes absorbance spectrum

0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

beforebeforeburntburnt

Photon energy (eV)

coun

ts

Damage changes absorbance spectrum

0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

beforebeforeburntburnt

Photon energy (eV)

coun

ts

1

0

fluorescence probe for damage

Absorbed Dose (MGy)

Fra

ctio

n u

nco

nve

rted

Wide range of decay rates seen

0.

0

0

.2

0.4

0.6

0

.8

1.0

0 50 100 150 200

Half-dose = 41.7 ± 4 MGy“GCN4” in crystal

Half-dose = 5.5 ± 0.6 MGy8 mM SeMet in NaOH

Protection factor: 660% ± 94%

http://ucxray.berkeley.edu/~jamesh/elves

Download Elves from: