Nature Methods: doi:10.1038/nmeth · 33.#RadialSymmetry# Contact: Raghuveer Parthasarathy,...

Nature Methods: doi:10.1038/nmeth.3442

Supplementary Figure 1

Comparative results for detection rate and accurary for the long sequence datasets.

Plot the detection rate (Jaccard) vs. the accuracy of every software and plot the efficiency line for every dataset. The crosses indicates the results of the lower-bound software (CenterOfGravity) that we have developped.



Comparative results for detection rate and accurary for the high-density datasets.

Plot the detection rate (Jaccard) vs. the accuracy of every software and plot the efficiency line for every dataset.



Correlation of the quantitative assessement criteria.

Plots of the 2 by 2 cross-correlation of the four quantitative criteria, detection rate (JAC), accuracy (ACC), image quality assessment (SNR), and image resolution (FRC). The results of all evaluated software are plotted in a different color for every dataset, the 3 long-sequence datasets (LS) and the high-density datasets (HD). The position of the crosses indicates the average per dataset and the length of its arms is equal to the standard deviation.

SMLM software based on explicit localization of single molecule

1. 3D-‐DAOSTORM Contact: Hazen Babcock, Harvard University (Zhuang group), Cambridge, MA, USA Reference: Hazen Babcock, Yaron Sigal and Xiaowei Zhuang, A high-density 3D localization algorithm for stochastic optical reconstruction microscopy, Optical Nanoscopy 1, 2012. Link: http://zhuang.harvard.edu/software/3d_daostorm.html Date: 2012 HD: Specifically designed for high-density datasets Challenge participation: Run by an expert

2. a-‐livePALM Contact: Yiming Li Institute of Applied Physics Karlsruhe Institute of Technology (KIT) Karlsruhe Germany Reference: Y. Li et al. Fast and efficient molecule detection in localization-based super-resolution microscopy by parallel adaptive histogram equalization, ACS Nano 7, 2013. Link: http://www.aph.kit.edu/nienhaus/english/26_116.php Date: 2013 Challenge participation: Run by the author

3. Auto-‐Bayes Contact: Yunqing Tang and Luru Dai, National Center for Nanoscience and Technology, Beijing, China Link: http://english.nanoctr.cas.cn/dai/software/ Date: 2015 Challenge participation: Run by the author

4. B-‐recs Contact: Herve Rouault and Dmitri Chklovskii, Janelia Farm Research Campus, HHMI, USA Date: 2013 Challenge participation: Run by the author

5. ClearPALM Contact: Arun Shivanandan, Laboratory of Nanoscale Biology, EPFL, Switzerland Date: 2007 Challenge participation: Registered software but not yet run

6. DAOSTORM Contact: Seamus J. Holden, Laboratory of Experimental Biophysics, EPFL, Switzerland Reference: S.J. Holden, S. Uphoff, A.N. Kapanidis, DAOSTORM: an algorithm for high density superresolution microscopy, Nature Methods 8, 2011. Link: http://www.physics.ox.ac.uk/users/kapanidis/group/Main.Software.html Date: 2011 HD: Specifically designed for high-density datasets Challenge participation: Run by the author

Supplementary Note 1


2

7. FALCON Contact: Junhong Min and Jong Chul Ye, Bio-Imaging and Signal Processing Lab, KAIST, Republic of Korea Reference: Junhong Min et al., FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data, Scientific Reports 4, 2014. Link: http://bispl.weebly.com/nanoscopy.html Date: 2014 HD: Specifically designed for high-density datasets Challenge participation: Run by the author

8. Fast-‐ML-‐HD Contact: Kyungsang Kim and Jong Chul Ye, Bio-Imaging and Signal Processing Lab, KAIST, Republic of Korea Reference: K.S. Kim et al., Fast maximum likelihood high-density low-SNR super-resolution localization microscopy, SampTA'13 Conference Bremen, 2013. Date: 2013 HD: Specifically designed for high-density datasets Challenge participation: Run by the author

9. FastSR Contact: Yanhua Wang and Leslie Ying, Department of Biomedical Engineering and Department of Electrical Engineering, University at Buffalo, USA Date: 2013 Challenge participation: Run by an expert

10. FPGA Contact: Manfred Kirchgessner, Frederik Gruell, Heidelberg University, Germany Reference: F. Gruell et al., Accelerating Image Analysis for Localization Microscopy with FPGAs, International Conference on Field Programmable Logic and Applications (FPL), 2011. Date: 2011 Challenge participation: Run by the author

11. Gauss2dcirc Contact: Stephen Anthony Reference: S.M. Anthony, S. Granick, Image analysis with rapid and accurate two-dimensional Gaussian fitting, Langmuir 25, 2009. Link: http://groups.mrl.uiuc.edu/granick/software.html Date: 2009 Challenge participation: Registered software but not yet run

12. GPUgaussMLE Contact: Keith Lidke and Bernd Rieger, Delft University of Technology, The Netherlands Reference: C.S. Smith et al., Fast, single-molecule localization that achieves theoretically minimum uncertainty, Nature Methods 7, 2010. Link: http://panda3.phys.unm.edu/~klidke/software.html Date: 2010 Challenge participation: Run by the author


3

13. GMIMPRO Reference: G. Mashanov, J. Molloy, Automatic Detection of Single Fluorophores in Live Cells, Biophysical Journal 92, 2007. Link: http://www.nimr.mrc.ac.uk/gmimpro/ Date: 2007

14. GraspJ Contact: Norman Brede and Melike Lakadamyali, ICFO-Institut de Ciencies Fotoniques, Barcelona, Spain Reference: N. Brede and M. Lakadamyali, GraspJ: an open source, real-time analysis package for super-resolution imaging, Optical Nanoscopy 1, 2012. Link: http://code.google.com/p/graspj/ Date: 2012 Challenge participation: Run by the author

15. Insight3 Contact: Ryan McGorty and Bo Huang, Dept of Pharmaceutical Chemistry, University of California, San Francisco, USA Link: http://huanglab.ucsf.edu/STORM.html Date: 2013 Challenge participation: Run by the author

16. JD Localization Contact: Josh Larkin, University of Oxford, Oxford, UK Reference: J. Larkin, P. Cook, Maximum precision closed-form solution for localizing diffraction-limited spots in noisy images, Optics Express 20, 2012. Date: 2012

17. livePALM Reference: P. N. Hedde et al., Online image analysis software for photoactivation localization microscopy, Nature Methods 6, 2009. Link: http://www.nature.com/nmeth/journal/v6/n10/suppinfo/nmeth1009-689_S1.html Date: 2009

18. Localizer / Igor Reference: P. Dedecker et al., Localizer: fast, accurate, open-source, and modular software package for superresolution microscopy, Journal of Biomedical Optics 17, 2012. Link: http://www.igorexchange.com/project/Localizer Date: 2012

19. Lucky Imaging Contact: Brid Cronin, Keble College, Oxford, UK Reference: B. Cronin, B. de Wet, M. Wallace, Lucky Imaging: Improved Localization Accuracy for Single Molecule Imaging, Biophysical Journal 96, 2009. Date: 2009


4

20. M2LE Contact: Shane Stahlheber and Alex Small, California State Polytechnic University, Pomona, USA Reference: R. Starr, S. Stahlheber and A. Small, Fast maximum likelihood algorithm for localization of fluorescent molecules, Optics Letters 37, 2012. Link: http://code.google.com/p/molecule-localization-plugin/ Date: 2012 Challenge participation: Registered software but not yet run

21. MaLiang Contact: Zhen-li Huang and Yi-na Wang, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology,Wuhan, P. R. China Reference: T. Quan et al., Ultra-fast, high-precision image analysis for localization-based super resolution microscopy, Optics Express 11, 2010. Link: http://bmp.hust.edu.cn/srm/ Date: 2010 Challenge participation: Run by the author

22. Micro-‐Manager LM Contact: Nico Stuurman, Vale Lab, University of California, San Francisco, USA Reference: A. Edelstein et al., Advanced methods of microscope control using Î¼Manager software, Journal of Biological Methods 1(2), 2014. Link: http://valelab.ucsf.edu/~MM/MMwiki/index.php/Localization_Microscopy Date: 2014 Challenge participation: Run by the author

23. MrSE Contact: Zhen-li Huang and Yi-na Wang, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology,Wuhan, P. R. China Reference: H. Ma, F. Long, S. Zeng, and Z.-L. Huang, Fast and precise algorithm based on maximum radial symmetry for single molecule localization, Optics Letters 37, 2012. Link: http://bmp.hust.edu.cn/srm/ Date: 2012 Challenge participation: Run by the author

24. MTT Reference: Arnauld Serge, Nicolas Bertaux, Herve Rigneault and Didier Marguet, Multiple-target tracing (MTT) algorithm probes molecular dynamics at cell surface, Nature Protocol Exchange, 2008. Date: 2008

25. Octane Reference: Lili Niu and Ji Yu, Investigating intracellular dynamics of FtsZ cytoskeleton with photoactivation single-molecule tracking, Biophysical Journal 4, 2008. Link: https://github.com/jiyuuchc/Octane Date: 2008 Challenge participation: Registered software but not yet run

26. palm3d Reference: A. York et al., Confined activation and subdiffractive localization enables whole-cell PALM with genetically expressed probes, Nature Methods 8, 2011. Link: http://code.google.com/p/palm3d/ Date: 2011


5

27. PALMER Contact: Zhen-li Huang and Yi-na Wang, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology,Wuhan, P. R. China Reference: Y. Wang, T. Quan, S. Zeng, Z.L. Huang, PALMER: a method capable of parallel localization of multiple emitters for high-density localization microscopy, Optics Express 14, 2012. Link: http://bmp.hust.edu.cn/srm/ Date: 2012 HD: Specifically designed for high-density datasets

28. PC-‐PALM Reference: P. Prabuddha, T. Jovanovic-Talisman, J. Lippincott-Schwartz, Quantifying spatial organization in point-localization superresolution images using pair correlation analysis, Nature Protocol Exchange 8, 2013. Date: 2013

29. PeakFit Contact: Alex Herbert, Genome Damage and Stability Centre, University of Sussex, UK Link: http://www.sussex.ac.uk/gdsc/intranet/microscopy/imagej/smlm_plugins Date: 2013 Challenge participation: Run by the author

30. PeakSelector Reference: G. Shtengel et al., Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure, PNAS 9, 2009. Date: 2009 Challenge participation: Registered software but not yet run

31. PYME Contact: David Baddeley, Nanobiology Institute, Yale University, West Haven, USA and University of Auckland, Auckland, New Zealand Reference: D. Baddeley, M.B. Cannell, C. Soeller, Three-dimensional sub-100 nm super-resolution imaging of biological samples using a phase ramp in the objective pupil, Nano Research, 2011. Link: http://code.google.com/p/python-microscopy Date: 2011 Challenge participation: Run by the author

32. QuickPALM Contact: Ricardo Henriques and Christophe Zimmer, Institut Pasteur, Paris, France Reference: R. Henriques et al., QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ, Nature Methods 7, 2010. Link: https://code.google.com/p/quickpalm/ Date: 2010 Challenge participation: Run by the author

33. RadialSymmetry Contact: Raghuveer Parthasarathy, Department of Physics, The University of Oregon, USA Reference: R. Parthasarathy, Rapid, accurate particle tracking by calculation of radial symmetry centers, Nature Methods 9, 2012. Link: http://physics.uoregon.edu/~raghu/particle_tracking.html Date: 2012 Challenge participation: Run by the author


6

34. RainSTORM Reference: M. Erdelyi, E. Rees, D. Metcalf, G.S.Schierle, L. Dudas L et al., Correcting chromatic offset in multicolor super-resolution localization microscopy, Opt Express 21, 2013. Link: http://laser.cheng.cam.ac.uk/wiki/index.php/Resources Date: 2013

35. RapidSTORM Contact: Steve Wolter, Markus Sauer, Google Inc. and Biotechnologie and Biophysik, University of Wurzburg, Germany Reference: S. Wolter et al., rapidSTORM: accurate, fast open-source software for localization microscopy, Nature Methods 9, 2012. Link: http://www.super-resolution.biozentrum.uni-wuerzburg.de/research_topics/rapidstorm/ Date: 2012 Challenge participation: Run by the author

36. SHRImP Reference: M. Gordon, T. Ha, P. Selvin, Single-molecule high-resolution imaging with photobleaching, PNAS 101, 2004. Date: 2004

37. SimplePALM Contact: Jerome Boulanger and Leila Muresan, Institut Curie, Paris, and Centre de genetique moleculaire, Gif-sur-Yvette, France Reference: J. Boulanger and al., Patch-based non-local functional for denoising fluorescence microscopy image sequences, IEEE Trans. on Medical Imaging 29, 2010. Date: 2010 Challenge participation: Run by the author

38. SimpleSTORM Contact: Ullrich Koethe and Luca Fiaschi, Heidelberg Collaboratory for Image Processing, Germany Reference: U. Koethe, F. Herrmannsdoerfer, I. Kats, F.A. Hamprecht, SimpleSTORM: a fast, self-calibrating reconstruction algorithm for localization microscopy, Histochemistry and Cell Biology, 2014. Link: https://github.com/ukoethe/simple-STORM.git Date: 2014 Challenge participation: Run by the author

39. SNSMIL Contact: Yunqing Tang, Luru Dai, Chongqing University, Chongqing and National Center for Nanoscience and Technology, Beijing, P.R. China Link: http://english.nanoctr.cas.cn/dai/software/ Date: 2013 Challenge participation: Run by the author

40. SOSplugin Contact: Ihor Smal Erik Meijering, Erasmus MC - University Medical Center Rotterdam, the Netherlands Link: http://smal.ws/home/Software Date: 2013 Challenge participation: Run by the author


7

41. SparseSR Contact: Gongguo Tong, University of Wisconsin, Madison, USA Date: 2013 Challenge participation: Run by an expert

42. ThunderSTORM Contact: Guy Hagen, Pavel Krizek, Charles University, Prague, Czech Republic Reference: M. Ovesny and al., ThunderSTORM: a comprehensive ImageJ plugin for PALM and STORM data analysis and super-resolution imaging, Bioinformatics, 2014. Link: http://code.google.com/p/thunder-storm/ Date: 2014 Challenge participation: Run by the author

43. Wavelet FluoroBancroft Contact: Sean B. Andersson, Trevor Ashley, Boston University, USA Reference: S.B. Andersson, Localization of a fluorescent source without numerical fitting, Optics Express 16, 2008. Link: http://www.bu.edu/anderssonlab/ Date: 2008 Challenge participation: Run by the author

44. WaveTracer Contact: Adel Kechkar and Jean-Baptiste Sibarita, University of Bordeaux and Interdisciplinary Institute for Neuroscience, Bordeaux, France Reference: A. Kechkar, D. Nair, M. Heilemann, D. Choquet, J.-B. Sibarita, Real-time analysis and visualization for single-molecule based super-resolution microscopy, PLoS One 8, 2013. Date: 2013 Challenge participation: Run by the author

45. Wedged Template Matching Contact: Shigeo Watanabe and Keith Bennett, Hamamatsu Photonics K.K., Japan Reference: S. Watanabe et al, Evaluation of Localization Algorithm of High-Density Fluorophores, Wedged Template Matching, Focus on Microscopy Conference (FOM'13) Maastricht, 2013. Date: 2013 HD: Specifically designed for high-density datasets Challenge participation: Run by the authot


8

SMLM software based on deconvolution methods

46. 3B Reference: E. Rosten,G. Jones, S. Cox, ImageJ plug-in for Bayesian analysis of blinking and bleaching, Nature Methods 10, 2013. Link: http://www.coxphysics.com/3b/ Date: 2013

47. CSSTORM/FasterSTORM Reference: L. Zhu, W. Zhang, D. Elnatan, B. Huang, FasterSTORM using compressed sensing, Nature Methods 9, 2012. Link: http://www.nature.com/nmeth/journal/v9/n7/abs/nmeth.1978.html Date: 2012 HD: Specifically designed for high-density datasets Challenge participation: Registered software but not yet run

48. DeconSTORM Contact: Eran Mukamel, Havard University (Zhuang group), Cambridge, MA, USA Reference: E. A. Mukamel, H. Babcock, X. Zhuang, Statistical Deconvolution for Superresolution Fluorescence Microscopy, Biophysical Journal 102, 2012. Link: http://zhuang.harvard.edu/software/decon_storm.html Date: 2012

49. FacePALM Contact: Eelco Hoogendoorn, Molecular Cytology, Swammerdam Institute for Life Sciences, University of Amsterdam Reference: E. Hoogendoorn et al., Image deconvolution techniques with robust background estimation applied to photactivation localization microscopy (PALM), Focus On Microscopy Conference (FOM'13), Maastricht, The Netherlands, 2013. Date: 2013 Challenge participation: Run by the author

50. iNMF Contact: Ondrej Mandula, Fondation Nanosciences, Grenoble, France Reference: O. Mandula et al., Localisation microscopy with quantum dots using non-negative matrix factorisation, Optics express 22.20, 2014. Link: https://github.com/aludnam/inmf Date: 2014

51. L1H Contact: Hazen Babcock, Harvard University (Zhuang group), Cambridge, MA, USA Reference: Hazen Babcock, Jeff Moffit, Yunlong Cao and Xiaowei Zhuang, Fast compressed sensing analysis for super-resolution imaging using L1-homotopy, Optics Express 21, 2013. Link: http://zhuang.harvard.edu/software/l1h.html Date: 2013 HD: Specifically designed for high-density datasets Challenge participation: Run by an expert


The following abstracts were provided by the authors of the software and the participants of the IEEE ISBI Grand Challenge 2013.

a-‐livePALM Yiming Li1, Yuji Ishitsuka1, Per Nikals Hedde1 and G. Ulrich Nienhaus1,2,3

1Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany 2Department of Physics, University of Illinois at Urbana-Champaign, Urbana, Illinois , USA

3Karlsruhe Institute of Technology (KIT), Eggenstein-Leopoldshafen, Germany

Fast and efficient molecule detection in localization-‐based super-‐resolution microscopy by parallel adaptive histogram equalization

In localization-based super-resolution microscopy, individual fluorescent markers are stochastically photoactivated and subsequently localized within a series of camera frames, yielding a final image with a resolution far beyond the diffraction limit. Yet, before localization can be performed, the sub-regions within the frames where the individual molecules are present have to be identified, most often in the presence of high background. In this work, we present a fast and robust molecule detection algorithm that utilizes parallel application of adaptive histogram equalization to probabilistically identify molecule candidates even in the presence of heterogeneous background. We evaluated the performance of our new algorithm and compared the results to the search algorithm widely used in localization microscopy on both simulated and experimental data. We found that our new algorithm can identify molecule candidates with higher efficiency and reliability over a wide range of background conditions.

B-‐Recs Hervé Rouault, Dmitri Chklovskii

Janelia Farm Research Campus, HHMI, USA

B-‐recs: an efficient Bayesian algorithm for high density localization microscopy reconstruction

Single-molecule localization microscopy, such as PALM or STORM, relies on the sequential activation of bright fluorophores. If the density of molecules activated in each frame is small, they generate isolated responses on the camera. Knowing the point spread function (PSF) of the imaging system, it is possible to determine the number and positions of active fluorophores with high accuracy. At higher densities, the responses of the fluorophores registered by the camera overlap and photon noise as well as variations in fluorophore brightness impede the determination of the number of active fluorophores and their respective locations. This leads to a well-posed but difficult to solve high-dimensional probabilistic problem. We introduced an approximate yet efficient way to solve this problem using a Bayesian inference framework and developed a software package called B-recs. In B-recs, the prior on fluorophore intensity as well as the PSF can be chosen arbitrarily. For each acquired frame, B-recs outputs a high-resolution image representing the estimated intensity of the fluorophores with either the minimum mean squared error (MMSE) or the maximum a posteriori (MAP) estimate. For super-resolution images, we rely on MMSE estimate because it represents the degree of uncertainty by the spread of fluorophore location. Moreover, MMSE estimate allows reconstructing subdiffraction images at high densities for which the accurate determination of the number of fluorophores is impossible. However, when fluorophore localization is required, the MAP estimate can be output. B-recs allows one to choose a compromise between image acquisition time, spatial resolution and temporal resolution.



2

DAOSTORM Seamus J. Holden

Laboratory of Experimental Biophysics, EPFL, Switzerland

DAOSTORM was the first algorithm designed specifically for the analysis of high density super resolution microscopy data. The key advance which allows high density analysis is to simultaneuosly fit multiple model PSFs to the data, rather than just a single model PSF.

S. J. Holden, S. Uphoff, and A.N. Kapanidis, DAOSTORM: an algorithm for high density super-resolution microscopy, Nature Methods, 2011, 8, 279-280.

FacePALM Eelco Hoogendoorn, Kevin C. Crosby, Theodorus J.W. Gadella, Marten Postma

Molecular Cytology, Swammerdam Institute for Life Sciences, University of Amsterdam

FacePALM is a deconvolution based method for performing stochastic super resolution reconstructions. We emphasize reliability of the reconstruction over a wide spectrum of data qualities, including extreme event densities and complex backgrounds. By carefully solving the inverse imaging problem, systematic bias in the reconstruction due to high event densities can be avoided. Complex structured backgrounds are estimated and accounted for. This combination of techniques guarantees a monotonous relationship between photons emitted by the sample and photons included in the reconstruction, precluding the appearance of false contrast. By means of improving upon state of the art invers imaging solvers, and providing a GPU implementation, analysis times are reduced by several orders of magnitude, permitting the processing of several 512x512 frames per second on a single consumer GPU.

Fast-‐ML-‐HD Kyungsang Kim, Jong Chul Ye

Bio-Imaging and Signal Processing Lab, KAIST, Republic of Korea

Fast maximum likelihood high-‐density low-‐SNR super-‐resolution localization microscopy

Localization microscopy such as STORM/PALM achieves the super-resolution by sparsely activating photo-switchable probes. To use conventional algorithms, however, only small set of probes need to be activated simultaneously, which limits the temporal resolution. Hence, to improve temporal resolution up to a level of live cell imaging, high-density imaging algorithms that can resolve several overlapping PSFs are required. Therefore, we developed a maximum likelihood algorithm under Poisson noise model for the high-density low-SNR STORM/PALM imaging using a sparsity promoting prior with a concave-convex procedure (CCCP) optimization algorithm. We achieved high performance reconstructions with ultra-fast reconstruction speed of less than 10 seconds per frame under high density low SNR imaging conditions.

FPGA Manfred Kirchgessner1, Frederik Grüll2

1Heidelberg University, Germany 2Goethe University, Frankfurt am Main, Germany

Parallel localization analysis based on Gaussian estimator

We present a software based on the Gaussian Estimator for the analysis of imagery from Localization Microscopy. It was developed to accelerate the computational analysis for Localization Microscopy towards real-time. Initially, the design was chosen to enable an implementation on reconfigurable hardware. Field programmable hardware (FPGAs) accelerate applications especially well when the architecture closely follows a pipelined data-flow paradigm, giving the Gaussian Estimator an advantage over a least-square fit. Starting from a maximum-likelihood fit, we applied background removal with exponential smoothing as a first step. The fit for the remaining signals can then be solved analytically and results in the Gaussian Estimator. Additional steps have been added to suppress the confusion of background noise with signals and


3

to separate slightly overlapping signals. It speeds up analysis by two orders in magnitude compared to a least-square fit on a CPU while maintaining full accuracy up to few nanometers. It eventually turned out that the FPGA implementation, which accelerated analysis by an additional factor of 225, was not required to achieve real-time processing times. We ported the initial Matlab implementation into a parallelized C++/Qt program with a graphical user interface that utilizes modern multi-core CPUs. Every step in the algorithm was mapped to one or more threads of execution. As a result, the implementation is fast enough to calculate each data set from the competition in less than 60 seconds on an AMD Phenom II 965 CPU at 3.4 GHz.

GPUgaussMLE Robert P.J. Nieuwenhuizen1, Carlas S. Smith1,2,Keith A. Lidke3 and Bernd Rieger1

1Delft University of Technology, Delft, The Netherlands 2University of Massachusetts, Worcester, MA, USA

3Dept. of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA

Our algorithm produces a maximum likelihood estimate of the fluorophores' lateral positions, peak intensity, background level and Gaussian width using a Gaussian point spread function (PSF) model and was first described in Smith et al. The algorithm starts by loading the raw image data and subtracting either a fix offset value or a dark image from all frames. Subsequently, the pixel values are converted into photon counts by dividing them by the gain. The algorithm next searches for candidate positions of emitters, using the method described in Huang et al. In brief, this method applies two uniform filters of different kernel sizes to the data to search for regions of elevated intensity relative to the local background. The algorithm then takes regions of pixels around the candidate emitter positions and computes on the GPU a maximum likelihood estimate of the emitters' positions, assuming only Poisson statistics for the noise. The algorithm also estimates the emitters' intensities, the local background photon count and the PSF width, as well as the localization uncertainty of each parameter. Finally, the localizations are filtered based on these outcomes. By default, localizations with fewer than 100 signal photons or more than 500 background photons per pixel are rejected, as well as localizations where the estimated PSF width is more than 50% off from the expected value (100% for dataset 3LS). In addition a manual threshold was set on the estimated localization uncertainty, which was set such that images of deleted localizations had no apparent spatial information about the sample left. In Smith et al., it was shown that the algorithm achieves the highest localization precision that is theoretically possible in 2D when the noise follows Poisson statistics. This condition is not met for the datasets in this challenge due to the unknown EM gain which, moreover, was atypically low in the training datasets. As a result of this the read-out noise is non-negligible for these data and the pixel values in the raw data cannot be converted into photon counts. Therefore the algorithm will not perform as well for the datasets in this challenge as it would under typical imaging conditions, where an EMCCD camera is used with a high gain which has been calibrated.

C.S. Smith, N. Joseph, B. Rieger, and K.A. Lidke. Fast, single-molecule localization that achieves theoretically minimum uncertainty. Nature Methods, 7, 373-375 (2010).

F. Huang, S.L. Schwartz, J.M. Byars, and K.A. Lidke. Simultaneous multiple-emitter fitting for single molecule super-resolution imaging. Biomedical Optics Express, 2, 1377-1393 (2011).

GraspJ Norman Brede1,2, Melike Lakadamyali2

1Dept. of Physics and Astronomy, University of Heidelberg, Heidelberg, Germany 2DICFO-Institut de Ciencies Fotoniques, Mediterranean Technology Park, Barcelona, Spain

GraspJ (GPU-Run Analysis for STORM and PALM) is an open source data analysis and rendering tool for super-resolution imaging based on single molecule localization. GraspJ uses the Maximum Likelihood Estimation with a Gaussian point spread function (PSF) and a Poisson noise model to localize the x-y positions of single molecules with high precision. In 3D super-resolution images based on astigmatism, GraspJ can also localize the z-positions of single molecules from the PSF widths in x-y. For multi-color STORM imaging based on the use of separate activator dyes, GraspJ automatically splits the single molecule localizations into different color-channels based on the imaging cycle. Furthermore, GraspJ contains other functionalities essential for obtaining a final super-resolution image: drift correction (based on correlating images generated from subsets of frames or fiduciary markers) and high resolution rendering (using


4

Gaussians with widths that correspond to the localization precision and z-color-coding for 3D representation). GraspJ runs in real-time since the computationally demanding parts are implemented on the computer's graphic card using OpenCL making it fast. GraspJ is compatible with a large number of data acquisition software since it is designed to read-out the data as it is being written onto the hard disk. Last but not least, GraspJ is an ImageJ plugin and provides a convenient user interface and easy interfacing with other image processing tools available via ImageJ. Overall, we believe that the combination of fast speed, compatibility with many data acquisition software and the ImageJ interface makes GraspJ a powerful tool for super-resolution data analysis.

InSight3 Ryan McGorty, Bo Huang

Department of Pharmaceutical Chemistry, University of California, San Francisco, USA

Insight3: single-‐molecule analysis software

InSight3 is a single-molecule image analysis software written in C++. It includes single-molecule FRET, single-particle tracking and single molecule localization microscopy modules. The localization microscopy module performs single-molecule fitting on raw camera frames, filters the results, and renders a high-resolution image. Prior to fitting, single-molecule candidates in each camera frame are identified as pixels of higher intensity than those in a surrounding 5x5 pixel window. An optional Fourier low pass filter can be applied to raw images to reduce the noise. These candidates are fit using either a least-square or maximum likelihood algorithm and a model for the point-spread function which can be either a simple Gaussian, elliptical Gaussian or a more realistic point-spread function. After fitting, height, width, and ellipticity thresholds are applied to reject non-single-molecule spots. Molecules that stay fluorescent in consecutive frames are averaged to produce higher localization precision. A number of other features make InSight3 a powerful and user-friendly program. For example, it can: analyze multichannel data acquired with multiple activation or emission wavelengths; calibrate and perform 3D localizations with introduced astigmatism; correct drift using either the resulting localizations or interspersed bright-field images; and track moving molecules. It also offers a set of visualization and statistical analysis routines for localization results. Additionally, the list of molecule locations can be manipulated with a custom scripting language which gives the user flexibility in analysis.

M2LE Alex Small, Shane Stahlheber

California State Polytechnic University, Pomona, USA

M2LE: A fast localization algorithm

Our localization algorithm is called M2LE, standing for 2nd Moments and Maximum Likelihood Estimation. A synopsis of the algorithm's procedure is: 1) Regions of interest (ROIs) are identified by finding pixels that are brighter than background by a user-defined threshold, and drawing windows around those pixels. Inhomogeneous background is accounted for by dividing the image into several blocks and finding the median photon count on a pixel (valid if the density of fluorophores is low). An ROI is typically 7x7. 2) Each ROI is subjected to an ellipticity test, by computing the second moments of the image about its centroid. Images with low ellipticity are passed along for further analysis. The ellipticity threshold is user-specified, and can be made intensity- dependent to account for the fact that even a single-molecule image with a low photon count is likely to have non-zero ellipticity due to random noise. 3) ROIs that pass the ellipticity test are sent to an MLE algorithm that fits a Gaussian to the image. The separable property of the Gaussian is used to speed up the calculation. The x coordinate is obtained by summing the photon counts in the columns of the image and then fitting a Gaussian that has the x coordinate but not the y coordinate as a parameter. The y coordinate is estimated by summing rows and then fitting a Gaussian that has the y coordinate but not the x coordinate as a parameter. 4) It will often turn out that several nearby pixels are above the thresholds, and so a window is drawn around each of them. The windows drawn around pixels that are farther from the actual molecular position are going to have off-center images, with some pixels that contain useful information (photon counts) being dropped. To address this problem, if several position


5

estimates are close together, the only one retained is the estimate that is closest to the center of the window that it was obtained from.

R. A. Starr, S. Stahlheber, and A. Small, Fast maximum likelihood algorithm for localization of fluorescent molecules. Optics Letters 37, 2012.

MicroManager Nico Stuurman

Vale Lab, Dept. of Cellular and Molecular Pharmacology, University of California San Francisco, USA

MicroManager: LM plugin

The Localization Microscopy plugin for Micro-Manager was developed to facilitate single molecule tracking and registration of 2-color images (at nm resolution) in our lab (the lab of Ron Vale at UCSF). It consists of two parts, a fitter and a data viewer. The fitter detects spots and fits these with a Gaussian model, results are stored in a dataset accessible from the Viewer. The Viewer has facilities for storing and re-loading datasets, visualizing tracks, registering multi-color datasets, and primitive visualization tools for super-resolution images. The code is written in Java, and uses fitting algorithms from the Apache foundation. Gaussian fits are executed in multiple simultaneous threads, and speeds of 10,000-20,000 on modest hardware (recent laptop computer) are easily attainable (speed scales quite linearly with the number of cores available, and real-time processing of super-resolution data should be quite easily attainable). User can specify a noise tolerance (difference in intensity between a local maximum and the average of the four corners) used in the spot finder, filters can be specified for intensity and width of the spots found. A manual for the plugin can be found at: http://valelab.ucsf.edu/~MM/MMwiki/index.php/Localization_Microscopy. The code contains some facilities for reading metadata from Micro-Manager files, taking out these dependencies would make it possible to run the code as an ImageJ plugin (i.e., without Micro-Manager).

PeakFit Alex D. Herbert, Thomas Etheridge, Anthony M. Carr}

Genome Damage and Stability Centre, University of Sussex, UK

Single-‐molecule localisation software for ImageJ

We present software for single-molecule localisation microscopy based on 2D Gaussian fitting. For each frame candidate spots are identified and then fit using a local region surrounding each candidate. An estimate of the PSF width is required which can be obtained from the wavelength and optical system used for acquisition, or estimated from the image. Candidates are identified using a difference of smoothing on the image followed by non-maximal suppression. Smoothing is done using a mean filter with a typical block size of 0.5 and 3 times the PSF peak-width at half-maximum (PWHM), where the second smoothing image is subtracted from the first. Peak candidates are processed in descending height order and a 2D Gaussian is fit to the peak using a typical region of 2 to 5 times the PWHM around the candidate. Fitting is performed using a non-linear least squares Levenberg-Marquardt method until convergence of the sum-of-squares or the maximum iterations is exceeded. Fitted spots are filtered using signal-to-noise, width and coordinate shift criteria. Processing is stopped when a consecutive number of fits fail.To account for high density samples neighbour peaks within the block region are included in the fit if they are within a fraction of the height of the candidate peak, typically 30%. If multiple peak fitting fails then single peak fitting is attempted. Additionally the candidate can be fit using a two peak model if the fit residuals show a skewed distribution in the quadrants around the centre. The doublet fit is selected if it improves the corrected Akaike Information Criterion (AIC). To prevent over-counting when fitting multiple peaks a check is made for duplicates using a distance criterion before adding to the results. The method is applicable to all types of localisation microscopy data and results are suitable input for filtering methods using structural models. Image frames are processed independently allowing multi-threaded operation. The software is written as a Java library and runs as a suite of plugins for the ImageJ image analysis program. Plugins are provided for fitting single images or an image series, drift correction and clustering, and are fully scriptable within the ImageJ macro language. Results can be output to file or a rendered image using various methods. A plugin is available to allow real-time display of super-resolution images by using the image acquisition engine within Micro-Manager.


6

PYME David Baddeley1,2

1Nanobiology Institute, Yale University, West Haven, CT, USA 2University of Auckland, Auckland, New Zealand

The PYthon (localisation) microscopy environment

The PYthon Microscopy Environment (PYME) is a python package which offers a comprehensive package for localisation microscopy and other forms of advanced widefield microscopy. PYME includes components for microscope control, real time data analysis, image reconstruction/visualisation, and postproccesing. A modular architecture allows generic processing tasks to be distributed over a cluster of machines (or amongst separate worker processes running on different cores of a single machine). The software offers both standard 2D Gaussian fitting of localisation data and 3D PSF based fitting. Both fitting methods use a Levenburg-Marquardt solver to solve the weighted least-squares problem with a Gaussian-Poisson noise mixture model which closely approximates the noise behaviour of modern EMCCD cameras. It is worth noting that due to the inclusion of the non-zero camera read noise in the model our weighted least squares implementation does not suffer from the problems encountered at low photon counts when using a naive poisson approximation in weighted least squares, and is no less accurate than a Poisson-ML approach on simulated data. The results presented here are the result of 2D fitting, in a single emitter regime. After fitting the data was filtered, restricting the point list to those points with an estimated localisation precision of better than 30nm, as well as a Gaussian width and intensity within reasonable ranges. The Gaussian width based filtering offers some rejection of erroneous multi-emitter localisations, but also restricts the depth of field of the reconstruction, as is apparent in some of the competition samples.

QuickPALM Ricardo Henriques

Computational Imaging and Modeling Group, Biologie cellulaire et infection, Institut Pasteur, France

QuickPALM is a high-speed 3D particle detection and localization plugin for ImageJ. It has been designed to analyze and reconstruct super-resolution datasets from data acquired by methods based on the photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) techniques. It is easy to use and features: high-speed analysis, 2D single-particle localization or 3D using astigmatism, drift correction, batch analysis via ImageJ scripting and integrated analysis parallel to the acquisition (e.g.: using micromanager). Internally, QuickPALM uses an adapted Hogbom CLEAN method for spot finding, followed by a modified center of mass algorithm to compute the spot position and parameters defining spot shape. A full description of the algorithm can be found in its original Nature Methods paper.

R. Henriques, M. Lelek, E.F. Fornasiero, F. Valtorta, C. Zimmer, and M.M. Mhlanga, QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ. Nature Methods 7, 2010.

RadialSymmetry Raghuveer Parthasarathy

Department of Physics, The University of Oregon, USA

Single fluorophore localization by calculation of radial symmetry centers

I assess the performance of radial-symmetry-based fluorophore localization for the analysis of super-resolution microscopy images. As described recently, this method makes use of the observation that in the absence of noise, lines along local intensity gradients of any radially symmetric intensity distribution will intersect the symmetry center and hence identify the emitting particle's location. With noise, we can estimate the center as the point of minimal squared distance to gradient lines, which is easily analytically calculated. This approach has several attributes: (i) it does not require knowledge of the point spread function, and is independent of its particular form; (ii) it is very fast, involving algebraic calculation rather than iterative numerical methods; and (iii) its accuracy is similar to that of maximum likelihood estimation, and near the Cramer-Rao bound on precision. Real (or realistically simulated) super-resolution microscopy images contain both isolated fluorophores and fluorophores in close proximity with overlapping intensity


7

distributions. The former are well localized. The latter are challenging, for many techniques. Localization via radial symmetry does not presently include a means of localizing overlapping molecules, which lowers overall accuracy. Assessing the degree of gradient line intersection, however, provides a criterion for rejecting neighborhoods containing multiple fluorophores. My overall analysis procedure consists of identifying regions of bright local maxima in filtered images, applying radial-symmetry-based localization to the unfiltered images in each region, and culling based on width and gradient-line distance, all of which can be performed using a MATLAB GUI.

R. Parthasarathy, Rapid, accurate particle tracking by calculation of radial symmetry centers. Nature Methods, 9, 724-726 (2012).

rapidSTORM Steve Wolter1, Markus Sauer2

1Google Inc. 2Biotechnologie & Biophysik Universität Würzburg

rapidSTORM: Fast and versatile software for localization microscopy

The rapidSTORM project is an open source evaluation tool that provides fast and highly configurable data processing for single-molecule localization microscopy such as dSTORM. It provides both two-dimensional and three-dimensional, multi-color data analysis as well as a wide range of filtering and image generation capabilities. rapidSTORM aims for a complete implementation of low-density single-molecule localization microscopy, is the most feature-rich open implementation and is open for contributors. It is stand-alone C++ software and can be easily built on any GNU/Linux workstation.

Steve Wolter, Mark Schüttpelz, Marko Tscherepanow, Sebastian van de Linde, Mike Heilemann, and Markus Sauer, Real-time computation of subdiffraction-resolution fluorescence images. Journal of Microscopy, 237. 12–22, 2010.

Steve Wolter, Ulrike Endesfelder, Sebastian van~de Linde, Mike Heilemann, and Markus Sauer, Measuring localization performance of super-resolution algorithms on very active samples. Optics Express, 19. 7020–7033, Apr 2011.

Steve Wolter, Anna Löschberger, Thorge Holm, Sarah Aufmkolk, Marie-Christine Dabauvalle, Sebastian van~de Linde, and Markus Sauer, rapidstorm: accurate, fast open-source software for localization microscopy. Nature Methods, 9. 1040–1041, Nov 2012.

SimplePALM Jérome Boulanger1 and Leila Muresan2

1Subcellular structure and cellular dynamics - Institut Curie, CNRS, Paris, France 2Centre de génétique moléculaire, CNRS, Gif-sur-Yvette, France

We propose a simple PALM image reconstruction method which require little input from the user. In order to take into account the characteristic of the noise, we estimate its parameters and apply a variance stabilization approach. Then we propose to use a Difference of Gaussian filter in order to localize the fluorescent dyes. The scale of the filter is adjusted by using the given wavelength, numerical aperture and pixel size of the detector. In order to decide of the presence of spots, we apply a threshold derived from a probability of false alarm which is the only input of the method. Local maxima are then used to initialize a mean-shift approach to estimate subpixel localization of the dyes.

J. Boulanger, J.-B. Sibarita, C. Kervrann, P. Bouthemy, Non-parametric regression for patch-based fluorescence microscopy image sequence denoising. Proc. IEEE Int. Symp. on Biomedical Imaging: from nano to macro (ISBI 08), Paris, May 2008.

J. Boulanger, C. Kervrann, P. Bouthemy, P. Elbau, J.-B. Sibarita, J. Salamero, Patch-based non-local functional for denoising fluorescence microscopy image sequences. IEEE Trans. on Medical Imaging 29, 2010.


8

simpleSTORM Ullrich Koethe, Luca Fiaschi, Frank Herrmannsdoerfer, Ilia Kats, Fred Hamprecht

Heidelberg Collaboratory for Image Processing, Germany

simpleSTORM was designed to produce good reconstructions efficiently and with as little user input as possible: Only sensitivity and desired resolution have to be specified, whereas all remaining parameters (e.g. noise level, PSF shape, background intensity) are automatically derived from the input data in a self-calibration phase. If desired, the automatic choices can still be overridden by the user. Specifically, the algorithm proceeds in these steps: 1) Robust estimation of the hardware gain and offset parameters 2) Linear intensity transform into unit gain and zero offset to make the noise approximately Poisson 3) Anscombe transform of the intensities to make the noise approximately normal 4) Dynamic background estimation and subtraction 5) Estimation of the FWHM of a Gaussian PSF (as in our submission) or the PSF itself 6) Matched filtering of each frame with the PSF 7) Statistical test to detect significant maxima according to specified sensitivity 8) Cubic spline interpolation of the detections to specified subpixel accuracy.

The sensitivity parameter trades off precision vs. recall. In a high-recall setting, one can optionally perform density-based post-processing to improve precision, as in our submission. Running times are given for the parallelized version of our algorithm running on two cores.

SNSMIL Yunqing Tang1, Luru Dai2

1Department of Physics, Chongqing University, Chongqing, P.R. China 2National Center for Nanoscience and Technology, Beijing, P.R. China

SNSMIL: An open source software for super-‐resolution fluorescence microscopy mage reconstruction

SNSMIL (Shot Noise based Single Molecule Identification and Localization) is a cross-platform software aims at single molecule identification and localization in photoswitching super resolution microscopy image analysis and reconstruction. It is an accurate open source software based on CPU or GPU that allows real-time processing (under development). Instead of process EMCCD intensity image directly, SNSMIL firstly recovers photon-electrons for each pixels according to the working principle of electron multiplying. After Gaussian smooth to suppress high-frequency noise, local signal analysis is applied mean to deal with inhomogeneous background that happens often in experimental reality. Rose criterion states minimal contrast-to-noise ratio (CNR) required to distinguish signal from background and remains core concept in SNSMIL. When a single emitter is identified, a 2-dimensional Gaussian fit is apply to localize its precise position. Fundamentally, localization precision or confidence is confined by photo-electron harvested for an emitter and degraded by sample drift or vibration. Therefore, all fitted emitters are future qualified with a user defined resolution limit which in principle cannot be determined by localization software itself. Emitters with localization precision worse than the limit are abandoned. It gives user a reliable collection of emitters for super resolution image reconstruction with required accuracy. SNSMIL maintains high hitting rate for emitters while keeps low fraction of fake positives even in relatively poor signal-to-noise ratio (SNR) conditions and offers a new tool for signal molecule localization.

SOSplugin Ihor Smal, Erik Meijering

Erasmus MC - University Medical Center Rotterdam, the Netherlands

EM-‐based multiple object localization in fluorescence microscopy imaging

Our team participated in the Localization Microscopy Challenge using software (plugin for ImageJ), which is further referred as SOSplugin. The software implements a set of subsequent image processing steps that lead to detection and localization of diffraction limited (in appearance) objects, which are typically encountered in modern single molecule imaging experiments. All the steps were implemented using Java programming language as a plugin for ImageJ, which supports multi-threading on modern multi-core CPUs. To efficiently localize particles of interest in the noisy images and deal with spurious detections, each frame


9

in a sequence is first converted to 8-bit gray-scale image and transformed using the multiscale isotropic undecimated wavelet transform, with three scales. Next, the wavelet coefficients of the second and third scales below a threshold (given below) are discarded, and the corresponding retained coefficients are summed to yield a reconstructed image with reduced noise and low-frequency background variations. Then, the local maxima in the resulting images are taken as the candidate positions. To discriminate between true particles and noise, a 2D Gaussian fitting algorithm is applied to the original images at these maxima, which yields for each candidate position its subpixel coordinates (x, y), amplitude I, and size, expressed as the standard deviation σ of the Gaussian fit. All positions for which Iin [Imin, Imax] and σ in [σ_min, σ_max] are then accepted as true particle positions. Typical values used for the parameters (depending on the SNR) were Imin = 20 - 25, Imax = 255 - 300, σ_min = 0.5, σ_max = 1.5, where σ_min and σ_max are specified in pixels, and thresholding of the wavelet coefficients in the preprocessing step was done at a level of 1.5 σ W, where W is the standard deviation of the wavelet coefficients for a given scale. Further, the localizations, for which the estimated was in the range [σ_max ɑ, σ_max] (where ɑ = 0.1) were considered as suspicious' larger blobs, which might consist of several closely located subresolution objects. For those locations we ran the Gaussian mixture fitting procedure, based on Expectation-Maximization algorithm using 3 - 5 components. As a result, all the estimated locations were stored using the file format provided by the organizer of the challenge. Using the position of the localized objects, the high-resolution (at a scale of 10 nm/pixel) reconstructed images were created.

ThunderSTORM Martin Ovesny, Josef Borkovec, Guy M. Hagen, Pavel Krizek

Institute of Cellular Biology and Pathology, Charles University, Prague, Czech Republic

We have applied the same general approach as in our publication. We examined each of the three training datasets Tubulins I, Tubulins II and Bundled Tubes Long Sequence by running our software with 40 different combinations of noise reducing filters and spot detectors. We calculated the F-1 score for each combination of filter and detector, and chose the method with the highest score. The best method proved to be a combination of wavelet-based filtering for noise reduction together with a watershed-based spot detector, as introduced by Izeddin et al. We then used this combination to process the three "Long Sequence" contest data sets. For the high density data sets, we used the compressed sensing approach as introduced by Zhu et al. along with some post-processing. This approach gives excellent results, but it is computationally intensive. For the contest data set "1HD" (128 x 128 pixels, 1000 frames) we used 5 Intel Core i7 CPUs (40 processor cores, 3 GHz) running for approximately two days.

P. Krizek, I. Raska, and G.M. Hagen, Minimizing Detection Errors in Single Molecule Localization Microscopy. Optics Express 19, 2011.

Wavelet-‐FluoroBancroft Sean B. Andersson1.2, Trevor T. Ashley1

1Department of Mechanical Engineering, Boston University, USA 2Division of Systems Engineering, Boston University, USA

Wavelet multiscale product segmentation with fluoroBancroft localization

The Wavelet Multiscale Product method is used to segment each image into regions of interest. To further increase the sensitivity beyond the originally prescribed hard-threshold, the nonzero components in the multiscale product are normalized by a Box-Cox transformation and only components above some multiple of standard deviations from the mean are selected. These regions are then grouped based on their relative distances and assigned a representative rectangular block of pixels on which sub-pixel localization will be performed. The fluoroBancroft algorithm is used to localize a single particle within each representative block of pixels. The software is coded entirely in MATLAB and frame-by-frame parallelization may be used to increase run-time performance.

S.B. Andersson, Localization of a fluorescent source without numerical fitting. Optics Express 16, 2008.


10

WaveTracer Adel Kechkar, Deepak Nair, Daniel Choquet, Jean-Baptiste Sibarita

University of Bordeaux, Interdisciplinary Institute for Neuroscience, Bordeaux, France Institut Interdisciplinaire de Neurosciences, CNRS UMR 5297, Bordeaux, France

WaveTracer: real-‐time reconstruction for single-‐molecule based super-‐resolution microscopy

New single molecule based super-resolution imaging techniques, such as PALM, (d)STORM, and GSDIM, require a heavy computer processing step. A typical acquisition is composed of few ten-thousands of images, from which up to a million molecules has to be localized to reconstruct the super-resolved image with a resolution below the diffraction limit of light microscopy. Most of the available tools require off-line processing of the data, which prevent the direct access to the super-resolution image directly on the microscope, and makes the overall technique slow. In this work, we propose a new method for the real-time processing and visualization of super-resolved images. First, in order to provide a fast and accurate localization of the molecules, we have developed and algorithm based on the wavelet decomposition of the images. This approach has shown its efficiency, in terms of accuracy and speed of computing, comparing to the conventional Gaussian fitting method. Second, we have implemented such algorithm on GPU technology in order to greatly speed-up the calculations, making them compatible with real-time computation. The real-time reconstruction of super-resolved image is accompanied with a direct feed-back on the laser power according to the required density of molecules. It allows optimizing the acquisition parameters in order to get the optimal molecule's density during the overall acquisition and obtain the best resolution in a minimum amount of time. A combination with Gaussian fitting enables a direct processing of 3D localization. This step is achieved by NLLS (Non-Linear Least Squares) minimization implemented on massively parallel GPU hardware architecture where each GPU processor computes a single molecule fitting. This solution makes all single molecule based super-resolution microscopy techniques much more attractive since it saves lot of bandwidth and gives the user a direct access to the super-resolution images, similar to other optical methods such as STED but using a much simpler microscopy setup.

WTM Shigeo Watanabe2, Keith Bennett1, Teruo Takahashi2, Tomochika Takeshima2

1Hamamatsu Corporation, Bridgewater, NJ, USA 2Systems Division, Hamamatsu Photonics K.K., Joko-cho, Hamamatsu City, Japan

Localization of High-‐Density Fluorophores using Wedged Template Matching

The core principle of localization microscopy is selectively exciting single non-adjacent molecules over time, allowing the sub-diffraction location of each molecule to be mathematically determined. This elegant approach has limits including the development and implementation of switchable fluorophores and the need for many raw image frames to produce meaningful reconstructions. Solving the issues associated with overlapping fluorophore point spread functions will open the door to larger application of localization microscopy techniques including live cell microscopy. To address these problems we have developed a localization microscopy image reconstruction algorithm with two key features: 1) implementation of dynamic background subtraction and 2) wedged template matching (WTM). This new algorithm has the capability to localize molecules with simultaneously overlapping emitters. We tested our algorithm using simulated and experimental data sets. Our results show super resolution structures can be seen in reconstructed images from a single frame.

T. W. Quan, H. Y. Zhu, X. M. Liu, Y. F. Liu, J. P. Ding, S. Q. Zeng and Z. L. Huang, High-density localization of active molecules using Structured Sparse Model and Bayesian Information Criterion. Optics Express, 19, 16963-16974 (2011)

S.M. Fullerton, K. Bennett, E. Toda and T. Takahashi, Camera Simulation Engine Enables Efficient System Optimization for Super-Resolution Imaging. Proc. SPIE, 8228, In press.


2

Parameters of the simulation

Parameter Admissible values Experiment Field of view 10 ... 40 µm Sample thickness 100 ... 1000 nm Duration of the experiment 100 ... 1000 s Frame rate 25 ... 50 Hz Number of frames 100 ... 20,000 Activation Extinction coeffcient (EC) of the dye 50,000 ... 300,000

Excitation wavelength 500 ... 800 nm Quantum yield (QE) 0.5 ... 1 Laser power 100 ... 500 W. cm-2

Non-uniform excitation Bumped function controlled the decay and the center Flux of photons 1021 ... 1022 photons . cm-2 . s-1

Number of activated fluorophores 105 ... 107

Average number of fluorophores per frame 3 ... 30 for LS, 100 ... 1000 for HD Lifetime model Profile of emission Constant, linear decay, or exponential decay Distribution over the frame Uniform Delay Δ Fixed, Gaussian distribution, or Rayleigh distribution Number of photons Fixed, Gaussian distribution, or Rayleigh distribution PSF Model XY-Gaussian/Z-Exponential or Gibson-Lanni Emission wavelength 500 ... 800 nm Numerical aperture (NA) 1.4 ... 1.5 Diffraction limit 200 ... 300 nm Offset of the focal plane 0 ... 500 nm Camera Quantum efficiency 0.5 ... 2 Resolution 64 ... 256 pixels Pixelsize 100 ... 150 nm A/D converter Gain, offset, saturation level, and quantization level Autofluorescence Localization of sources Around the sample or random Sources Controlled by the number of sources, z-position Evolution in time Static or Dynamic Noise Shot noise Poisson distribution Readout noise Gaussian distribution with 10 ... 100 electrons as standard

deviation EMCCD noise Multiplication the electrons count by sqrt(2)



2. Contest Dataset LS1 and HD1

Figure 1: Structure of sample. Depth-color coded representation of the central axis of the microtubules.

Figure 2: Left: Z-projection view of the fluorophores at 50 nm/pixel, field-of-view: 12.8x12.8 µm2. Right:Z-projection view of the fluorophores at 2.93 nm/pixel, field-of-view: 0.75x0.75 µm2.

3


Setup Feature Value

Camera Quantum efficiency (QE) 1.00 e-/Ph.Resolution 128 pixelsPixelsize 100.00 nmField of view 12800.00 nm

ADC Electron conversion - Gain 1.00 DN/e-Electron conversion - Offset 0.00 DNBaseline 100.00 DNSaturation 65535.00 DNQuantization 16-bit

PSF Wavelength 655.00 nmNumerical aperture (NA) 1.46Diffraction limit 224.32 nmOffset focal plane 0 nmXY function GaussianFWHM 224.32 nmZ function ExponentialFocal plane 1 x FWHM at 0.0 nmDefocus plane 2 x FWHM at 500.0 nm

Experiment Feature Value LS1 Value HD1

Sequence Duration time 500 s 50 s.Integration time 20 ms 20 msFramerate 50.00 Hz 50.00 HzNumber of frames 10000 1000

Density Volumetric density 799.89 7987.63 molecule/µm3

Imaging density 0.1767 1.7641 molecule/µm2/frame

Table 1: Characteristic of the simulation.

Spatial dispersion Signal Long sequence (LS1) High density (HD1)

Stats x y z Int. dNN SNRf dNN SNRf

nm nm nm DN nm dB nm dB

Max 12649.9 12407.2 247.7 9561.5 9006.1 62.5 2203.7 81.8Min 1369.4 792.3 53.2 384.9 0.9 0.2 0.3 -0.4Mean 7508.9 7826.7 146.3 4929.4 976.5 10.3 157.0 6.7StDev 3166.5 3033.1 44.6 986.2 770.8 5.6 159.4 5.4

Table 2: Statistics of the main features for datasets LS1 and HD1.

4


Figure 3: Example of frames. Top: 3 frames extracted from the LS dataset. Bottom: 3 frames extractedfrom the HD dataset

5


Figure 4: Distribution of the local SNRf around every fluorophore from 0 dB to 20 dB.

Figure 5: Distribution of distance dNN of the nearest fluorophore from 0 nm to 2000 nm.

6




7


Figure 7: Left: Z-projection view of the fluorophores at 50 nm/pixel, field-of-view: 19.2x19.2 µm2. Right:Z-projection view of the fluorophores at 1.432 nm/pixel, field-of-view: 0.384x0.384 µm2.

Setup Feature Value



PSF Wavelength 723.00 nmNumerical aperture (NA) 1.40Diffraction limit 258.21 nmOffset focal plane 0 nmModel Gibson and LanniRefractive index sample 1.00Refractive index immersion 1.50Offest working distance 0.00 nm


Sequence Duration time 600 s 30 s.Integration time 20 ms 20 msFramerate 50.00 Hz 50.00 HzNumber of frames 12000 600




8








9




10




11


Figure 12: Left: Z-projection view of the fluorophores at 50 nm/pixel, field-of-view: 20x20 µm2. Right:Z-projection view of the fluorophores at 3.0 nm/pixel, field-of-view: 0.4x0.4 µm2.

Setup Feature Value



PSF Wavelength 723.00 nmNumerical aperture (NA) 1.46Diffraction limit 247.60 nmOffset focal plane 0 nmModel Gibson and LanniRefractive index sample 1.00Refractive index immersion 1.50Offest working distance 0.00 nm


Sequence Duration time 300 s 21 s.Integration time 33.33 ms 33.33 msFramerate 30.00 Hz 30.00 HzNumber of frames 10000 700




12








13




14


Date post:	28-Jun-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Nature Methods: doi:10.1038/nmeth · 33.#RadialSymmetry# Contact: Raghuveer Parthasarathy,...

Documents