Computational Tutorial: Calcium Imaging Data Cell Extraction · 2017. 11. 27. · Computational...

Computational Tutorial: Calcium Imaging Data Cell Extraction

Pengcheng Zhou

July 12th, 2017

Department of Statistics Columbia University

Computational Tutorial: Calcium Imaging Data Cell Extraction, 07/12/20171


Outline

➤ Part I: Overview of methods for cell extraction General methods for calcium imaging data

Constrained Nonnegative Matrix Factorization for microEndoscopic data (CNMF-E)

➤ Part II: Exercises

2


➤ Part I: Overview of methods for source extraction General methods for calcium imaging data




Calcium Imaging

4

ARTICLEdoi:10.1038/nature12354

Ultrasensitive fluorescent proteins forimaging neuronal activityTsai-Wen Chen1, Trevor J. Wardill1{, Yi Sun1, Stefan R. Pulver1, Sabine L. Renninger2, Amy Baohan1,3, Eric R. Schreiter1,Rex A. Kerr1, Michael B. Orger2, Vivek Jayaraman1, Loren L. Looger1, Karel Svoboda1 & Douglas S. Kim1

Fluorescent calcium sensors are widely used to image neural activity. Using structure-based mutagenesis and neuron-basedscreening, we developed a family of ultrasensitive protein calcium sensors (GCaMP6) that outperformed other sensors incultured neurons and in zebrafish, flies and mice in vivo. In layer 2/3 pyramidal neurons of the mouse visual cortex, GCaMP6reliably detected single action potentials in neuronal somata and orientation-tuned synaptic calcium transients in individualdendritic spines. The orientation tuning of structurally persistent spines was largely stable over timescales of weeks.Orientation tuning averaged across spine populations predicted the tuning of their parent cell. Although the somata ofGABAergic neurons showed little orientation tuning, their dendrites included highly tuned dendritic segments (5–40-mmlong). GCaMP6 sensors thus provide new windows into the organization and dynamics of neural circuits over multiple spatialand temporal scales.

Neural activity causes rapid changes in intracellular free calcium1–4. Cal-cium imaging experiments have relied on this principle to track the acti-vity of neuronal populations5,6 and to probe excitation of small neuronsand neuronal microcompartments2,7–10. Genetically encoded protein sen-sors can be targeted to specific cell types2,9,11,12 for non-invasive imagingof identified neurons and neuronal compartments8,13–15 over chronictimescales6.

Calcium indicator proteins include the single fluorophore sensorGCaMP (refs 11, 16, 17) and several families of Förster resonanceenergy transfer based sensors18–22. However, none of these protein-based indicators have yet surpassed the sensitivity and speed of commonly

used synthetic calcium indicators (for example, Oregon Green Bapta-1-AM, OGB1-AM). Therefore, depending on the experimental goals, inves-tigators choose between sensitive synthetic indicators delivered by invasivechemical or physical methods, or less sensitive protein sensors deliveredby genetic methods.

Multiple rounds of structure-guided design have made GCaMPs themost widely used protein calcium sensors11,16,17. But past efforts inoptimizing GCaMPs and other indicators of neuronal function werelimited by the throughput of quantitative and physiologically relevantassays. Because neurons have unusually fast calcium dynamics andlow peak calcium accumulations4, sensors designed to probe neuronal

1Janelia Farm Research Campus, Howard Hughes Medical Institute, 19700 Helix Drive, Ashburn, Virginia 20147, USA. 2Champalimaud Neuroscience Programme, Champalimaud Centre for the Unknown,Avenida Brası́lia, Doca de Pedrouços, 1400-038, Lisbon, Portugal. 3Department of Neurobiology, University of California Los Angeles, Los Angeles, California 90095, USA. {Present address: MarineBiological Laboratory, Program in Sensory Physiology and Behavior, 7 MBL Street, Woods Hole, Massachusetts 02543, USA.

R392G

K78H

A317E

M378GT381R S383T

K379SM13Linker 1cpEGFPLinker 2CaM

0 1 2 30

5

a b

c

0

0.31 action potential

10 actionpotentials

5G

OGB1

6f6m6s

GCaMP3

cpEGFP CaMGCaMP5G K78 A317 M378 K379 T381 S383 R392GCaMP6s K78H T381R S383T R392GGCaMP6m M378G K379S T381R S383T R392GGCaMP6f A317E T381R S383T R392G

ΔF/F

0ΔF

/F0

1 10 1000.1

1

10

SN

R(fo

ld o

f GC

aMP

3)e

1 10 1000

1

2

3

Dec

ay ti

me,

t 1/2

(s)

f

1 10 1000.1

1

10

d

(fold

of G

CaM

P3)

ΔF/F

01 10 1000

0.2

0.4

0.6g

Number of action potentialsR

ise

time,

t pea

k (s

)

Time (s)

Resting fluorescence F0

1 action potential ΔF/F0

1AP10AP

160AP

5GOGB1

5G

OGB1

5G3

3

3

P-value

10–510–5Less

NSMore Decay time, t1/2 (10 action potentials)

Figure 1 | GCaMP mutagenesis and screening indissociated neurons. a, GCaMP structure27,51 andmutations in different GCaMP variants relative toGCaMP5G. b, Responses averaged across multipleneurons and wells for GCaMP3, 5G, 6f, 6m, 6s, andOGB1-AM. Top, fluorescence changes in responseto 1 action potential. Bottom, 10 action potentials.c, Screening results, 447 GCaMPs. Top,fluorescence change in response to 1 actionpotential (vertical bars, DF/F0; green bar, OGB1-AM, left; black bars, single GCaMP mutations; redbars, combinatorial mutations; blue, GCaMP6indicators) and significance values for differentaction potential stimuli (colour plot). Middle, halfdecay time after 10 action potentials. Bottom,resting fluorescence, F0 normalized to nuclearmCherry fluorescence. Red line, GCaMP3 level;green line, GCaMP5G level; blue line, OGB1-AMlevel. AP, action potential. d–g, Comparison ofGCaMP sensors and OGB1-AM as a function ofstimulus strength (colours as in b). d, Responseamplitude. e, Signal-to-noise ratio (SNR). f, Halfdecay time. g, Time to peak (after stimulus offset).Error bars correspond to s.e.m (n 5 300, 16, 8, 11,13 and 11 wells for GCaMP3, GCaMP5G, OGB1-AM, 6f, 6m and 6s, respectively).

1 8 J U L Y 2 0 1 3 | V O L 4 9 9 | N A T U R E | 2 9 5

Macmillan Publishers Limited. All rights reserved©2013

Chen et.al., Nature 2013

spiking activity

intracellular [Ca2+]

fluorescence intensity

fast, electrical

slow, optical


Chen et.al., Nature 2013


Cell Extraction Problem

6

t0

Movie frames

t1

t2

...

tN

Identified cells Activity traces

...

Time

Given a video data, identify all neurons and extract their temporal traces.


Two Approaches

➤ROI analysis ➤Matrix factorization approach

7


ROI analysis

8

1. draw neuron’s ROI manually or automatically 2. extract neural activity using the summed fluorescence

intensities within each ROI

Smith & Hauser, Nature Neuroscience, 2010


Problems

9

Spatial overlaps

Pnevmatikakis et.al., arXiv 2014


Problems

10

extra fluorescence signals within ROI

Zhou et.al., arXiv 2016


Matrix Factorization Approach

11

1. = + ++ +…

2.

+

= X

background noisedata neuron 1 neuron 2 neuron K

spatial-temporal activity

neuron shape temporal activity


=

3D video

T frames

d pi

xel

2D matrix

We represent a video data as a matrix



13

1. = + ++ +…

2.

+

= X

background noisedata neuron 1 neuron 2 neuron K

spatial-temporal activity

neuron shape temporal activity



14

d pi

xel

T frames



15

d pi

xel

T frames

1. each column of A is the spatial shape of one neuron 2. each row of C is the temporal activity of one neuron

Cell extraction problem can be solved by finding A & C.



➤ simultaneously identifying neurons and extracting their temporal traces.

➤ demixing spatially overlapped neural signals.

16

d pi

xel

T frames



17

1. Finding A & C is not easy.

2. It requires reasonable constraints to model variables.

3. The quality of cell extraction depends on - how realistic are added constraints - how well we can estimate A & C given these constraints.



18

1. PCA/ICA (Mukamel et al., Neuron 2009) - Neuron shapes and temporal components are spatially and

temporally independent.

2. CNMF (Pnevmatikakis et al., Neuron 2016) - Constrained Nonnegative Matrix Factorization (CNMF) - Simultaneous denoising, deconvolution, and demixing of Calcium

Imaging Data - to be discussed

Other formulations: SHMF (Diego & Hamprecht 2013); NMF (Maruyama et al. 2014); SSTD(Diego & Hamprecht 2014)


CNMF framework

19

Pnevmatikakis et al., Neuron 2016

November 28, 2016DRAFT

Table 4.1: Variables and notationsName Description Dimension

Y motion corrected video data Rd⇥T+

A spatial footprints of all neurons Rd⇥K+

C temporal activity of all neurons RK⇥T+

B background activity Rd⇥T+

E observation noise Rd⇥T+

d number of pixels integerT number of frames integerK number of neurons integer

matrix factorization problem successfully is a nontrivial problem. To analyze different datasets,CNMF requires specific constraints to the model and specialized procedures for performingmatrix factorization. The vanilla CNMF is optimized for 2-photon and light-sheet imaging data,where the background is weak and has simple spatiotemporally separable structures. However,microendoscopic data has large rapid fluctuating background, which hinders the direct applicationof the vanilla CNMF.

In this work, we extend the CNMF framework to address the strong fluctuations in backgroundfluorescences, and make it applicable to microendoscopic data. Like the proposed CNMF in [46],our extended CNMF for microEndoscopic data (CNMF-E) also has the capability of identifyingneurons with low signal-to-noise ratio (SNR) and simultaneously denoising, deconvolving anddemixing large-scale microendoscopic data.

4.2 CNMF-E modelIn Section 3.3.2, we discussed the CNMF framework, which formulates the source extractionproblem as a matrix factorization problem. This is a general framework and can be applied to awide variety of calcium imaging data. However, the successful application to different datasetsrequire specialized implementations. We also showed that the vanilla CNMF does not work formicroendoscopic data due to the large background fluctuations that could not be precisely modeledin the current implementation. In this section, we present our CNMF-E model to fix this issue.

Our proposed CNMF-E method takes the same framework as CNMF but it models thebackground in a different way. The proposed model is able to successfully capture the high-rankfeature of the background and avoid including the cellular signals into the background term B.The initialization step is also a big issue in applying the vanilla CNMF to microendoscopic data,and we will describe our solution in the next section (Section 4.3). As a complete pipeline forautomatically processing microendoscopic data, CNMF-E only requires the motion correctedvideo data as input and very few parameter selections.

Considering the multiple background sources, CNMF-E starts from modeling the backgroundas a high-rank matrix

B = DF + b0

· 1T = Bf +Bc, (4.1)

52

white noise


➤ Both the spatial and the temporal components are nonnegative

➤ The spatial components have localized and compact footprints

➤ The temporal activity of each neuron obeys the dynamics of calcium indicators

CNMF framework

spiking signal

➤ The spiking signals are sparse

Pnevmatikakis et al., Neuron 2016


CNMF framework

➤ It is a general framework for analyzing calcium imaging data. It provides a generative model of data.

➤ Implementations to different datasets may require extra constraints to model variables.

➤ Estimating A & C is a nontrivial optimization problem.

➤ Finding a reasonable solution needs a good initialization method.

21

See Pnevmatikakis et al., Neuron 2016 for successful applications of CNMF.


➤ Part I: Overview of methods for source extraction General methods for calcium imaging data




Microendoscope

1. Ghosh, K. K., Burns, L. D., Cocker, E. D., Nimmerjahn, A., Ziv, Y., El Gamal, A., & Schnitzer, M. J. (2011). Miniaturized integration of a fluorescence microscope. Nature methods, 8(10), 871-878.

2. https://www.youtube.com/watch?v=K6Iw6VESQEo

https://www.youtube.com/watch?v=K6Iw6VESQEo

Joint PhD Program in Neural Computation and Machine Learning, Thesis Defense, 12/02/201624

Neuroscientists

behavior neural activity


Microendoscopic Data

1. GCaMP6s labeled PFC neuron. 2. The data is recorded by Shanna Resendez in Garret Stuber’s lab in UNC.

➤ strong background with rapid fluctuations

➤ many severely overlapped neurons

➤ low signal-to-noise ratios (SNRs)


CNMF-E

26

Constrained Nonnegative Matrix Factorization for microEndoscopic data (E also stands for extension).

1. BG: background

Background


CNMF-E

27

Compared to the plain CNMF, CNMF-E contains following modifications

➤ A new background model ➤ A new algorithm for fitting model variables ➤ A new initialization method


Model of the background

CNMF

Precise estimation of the background is crucial for accurate extraction of neural signals.

➤ Microendoscopic data has multiple sources of the background fluctuations (high rank).

➤ Simply increasing the rank of the B may include neural signals into the background.


Model of the background

neuron

fluctuating background + constant baselines

The background fluctuation at each pixel can be represented as a linear combination of its neighboring pixels’ fluctuations.

CNMF-E


The background model in CNMF-E can accurately recover the true background fluctuations in simulated data.



When number of background (BG) sources increases, CNMF-E can still estimate the background component B with high accuracy.

1. Mean corr. : mean correlation coefficient between the true background fluctuation and the estimated background fluctuation at each pixel.



The results of higher rank NMF model are still inferior to CNMF-E.



Solving CNMF-ESummary of the model

background

neuron


Optimization problem

non-convex


Three subproblems

November 29, 2016DRAFT

contributes as a very small disturbance to our estimated ˆBf , which is the left-hand side of Eq.(4.4). Here we optimize the weight matrix W first and then estimate Bf as W ·(Y �AC�b

0

·1T ),instead of optimizing Bf directly.

The problem (P-A) optimizes all variables together and is non-convex, which is notoriouslyhard to solve. To get an efficient solution, we divide it into three subproblems and then solve themiteratively.

Estimating A given ˆC, ˆBf , ˆb0

:

min

A

kY � A · ˆC � ˆb0

· 1T � ˆBfk2F

(P-S)

s.t. A � 0 and A is sparse.

Estimating C given ˆBf , ˆb0

, Â:

min

C

kY � Â · C � ˆb0

· 1T � ˆBfk2F

(P-T)

s.t. ci

� 0, si

� 0G(i)c

i

= s

i

, si

is sparse 8i = 1 . . . K

Estimating Bf , b0

given Â, ˆC

min

W,b0

kY � Â · ˆC � b0

· 1T � Bfk2F

(P-B)

s.t. Bf · 1 = 0Bf = W · (Y � Â · ˆC � b

0

· 1T ).W

ij

= 0 if dist(i, j) /2 (l, l + 1]

For each of these subproblems, we are able to use the well-established algorithms directly orslightly modify them to fit our needs. By iteratively solving these three subproblems, we gettractable updates for all the model variables in problem (P-A). Furthermore, this strategy givesus the flexibility of interfering the optimization procedure manually or automatically during themodel fitting procedure, such as merging/splitting/deleting the detected components and pickingneurons from the residuals. These steps can essentially improve the quality of the model fitting.

In these problems, (P-S) and (P-T) already have several efficient algorithms to solve [46, 109].In Appendices B.2 and B.3, we present details of the algorithms used in the current version ofthe CNMF-E implementation. These algorithms are specialized for different constraints added tothe sparsities of A and C. For some constraints that have not been proposed previously, we alsodevelop our customized algorithms to solve.

In the following, we mainly discuss our algorithm to estimate the background by solvingproblem (P-B) as a linear regression problem given Â and ˆC. We define X = Y � Â · ˆC, thenletting

ˆ

b

0

=

1

T(Y � Â · ˆC) · 1 = ¯X (4.5)

satisfies the constraint Bf · 1 = 0. We replace the b0

in (P-B) with this estimation and rewrite the

55

December 2, 2016DRAFT

Estimating A given ˆC, ˆBf , ˆb0

:

min

A

kY � A · ˆC � ˆb0

· 1T � ˆBfk2F

(P-S)

s.t. A � 0 and A is sparse.

Estimating C given ˆBf , ˆb0

,

ˆ

A:

min

C

kY � Â · C � ˆb0

· 1T � ˆBfk2F

(P-T)

s.t. ci

� 0, si

� 0G

(i)

c

i

= s

i

, s

i

is sparse 8i = 1 . . . K

Estimating Bf , b0

given Â, ˆC

min

W,b0

kY � Â · ˆC � b0

· 1T � Bfk2F

(P-B)

s.t. Bf · 1 = 0B

f

= W · (Y � Â · ˆC � b0

· 1T ).

For each of these subproblems, we are able to use the well-established algorithms directly orslightly modify them to fit our needs. By iteratively solving these three subproblems, we gettractable updates for all the model variables in problem (P-A). Furthermore, this strategy givesus the flexibility of interfering the optimization procedure manually or automatically during themodel fitting procedure, such as merging/splitting/deleting the detected components and pickingneurons from the residuals. These steps can essentially improve the quality of the model fitting.

In these problems, (P-S) and (P-T) already have several efficient algorithms to solve [44, 52].In Appendices B.2 and B.3, we present details of the algorithms used in the current version ofthe CNMF-E implementation. These algorithms are specialized for different constraints added tothe sparsities of A and C. For some constraints that have not been proposed previously, we alsodevelop our customized algorithms to solve.

In the following, we mainly discuss our algorithm to estimate the background by solvingproblem (P-B) as a linear regression problem given Â and ˆC. We define X = Y � Â · ˆC, thenletting

ˆ

b

0

=

1

T

(Y � Â · ˆC) · 1 = ¯X (2.8)

satisfies the constraint Bf · 1 = 0. We replace the b0

in (P-B) with this estimation and rewrite the(P-B) as

min

W

k ˜X �W · ˜Xk2F

, (P-W)

s.t. Wij

= 0 if d(xi

,x

j

) 6= l,

where ˜X = X � ¯X · 1T . Given the optimized ˆW , we make the estimation of the fluctuatingbackground as ˆBf = ˆW ˜X . The new optimization problem (P-W) can be readily parallelized into

19



Three subproblems


This strategy gives us the flexibility of including further potential interventions (either automatic or semi-manual) in the optimization procedure.


Interventions

➤ merge components into one neuron (automatically)

merge neurons with high temporal correlation

merge neurons whose centers are too close

➤ split a component into multiple neurons

➤ remove false positive components

➤ pick neurons from residuals (automatically)

CNMF-E allows many interventions during the model fitting. More iterations following the interventions produces better results.


Initialization



Example


Pipeline

initialize A & C

update B given A & C

interventions

update A given B & C

update C given A & C

1. A: spatial filter of all neurons 2. B: background activity 3. C: temporal activity of all neurons


Results 1: Simulation

41

CNMF-E recovers neural signals with high accuracy; CNMF-E preserves the pair-wise correlation between neurons.

1. We use the ground truth as initialization of CNMF method.


Example 1: Simulation

42

CNMF-E is robust to noise compared to PCA/ICA analysis.


Example 1: Simulation

43

similarity=0.598

similarity=0.866similarity=0.988

ground truth (black);PCA/ICA trace (blue); CNMF-E trace (red); CNMF-E trace without being denoised (cyan)


Example 2: in vivo PFC data

44

Raw data Identified neurons




45




46




47

The temporal traces extracted by CNMF-E have higher signal-to-noise ratios.



Example 3: in vivo BNST data

48


1. BNST: bed nucleus of the stria terminals



49


Dashed lines indicate electrical footshocks.



50


1. MAD: median absolute deviation, defined as the median of the absolute deviations from the data's median

Downstream analyses can significantly benefit from the improvements in the accuracy of source extraction


Summary

➤ Analyzing microendoscopic video data using CNMF-E

improve the accuracy of cell extraction

downstream analysis of calcium imaging data can significantly benefit from these improvements.

51


Summary➤ General calcium imaging data analysis

ROI analysis is simple and straightforward, but has limited performances.

CNMF is a general framework for simultaneously denoising, deconvolving and demixing calcium imaging data. Other matrix factorization methods are available as well (e.g., PCA/ICA, NMF, SHMF, SSTD).

Successful application of CNMF requires sufficient understanding of your data and modifying the model to fit your needs.

52


Part II: Exercises


Install CNMF-E

54

https://github.com/zhoupc/CNMF_E

https://github.com/zhoupc/CNMF_E


open demo script

55


select data

56

1. you can set the variable nam as the full path of your tif data.

2. If you didn’t create a variable nam for storing file path, CNMF-E will ask you to select a file interactively.

When you load a tif data for the first time, CNMF-E will save the data into *.mat file. Then CNMF-E will always load *.mat data, instead of tif data.


selection of gSig and gSiz

57

gSiz

gSiz/4


downsampling data for fast initialization

58

Run CNMF-E on down sampled data, then up-sample the results and use them as starting point of running CNMF-E on full resolution data.


merging thresholds

59

1. merge neurons with high temporal correlation (merge_thr, cnmfe_quick_merge).

2. merge neurons that are too close (dmin, cnmfe_merge_neighbors).

I usually use the second method in the last iteration.


options for running temporal deconvolution

60

When the rising time of one calcium transient is less than 1 frame, use ‘ar1’ type; otherwise, use ‘ar2’;

See Friedrich et al., Fast online deconvolution of calcium imaging data, PLOS Comp. Bio. (2017) for details of deconvolution algorithm.


load data

61

sframe: index of the first frame num2read: number of frames to be loaded.


compute correlation image and PNR image

62

pick some weak neurons and then use data curves to see their correlation values and PNR values.


Initialization

63

patch_par: divide the FOV into multiple patches when FOV is too large.


parameters for estimating background

64

spatial_ds_factor: estimating B using fewer pixels. bg_neuron_ratio: k, the radius of the ring / gSiz

neuron


three subproblems

65


update C given A & B



Interventions

66

display each neurons and allow you delete/split/trim neurons.

pick neurons from the residuals.

merge neurons.


Interventions

67

run CNMF-E on data with full resolution.


initialize A & C


interventions


update C given A & C


Future plans

69

- A complete suite for processing calcium imaging data, including motion correction as well.

- Support running CNMF-E on clusters for fast parallel computing

- Optimize algorithms for running CNMF-E in batch model and remove the bottleneck of memory usage.

- Allow running CNMF-E in real-time for closed-loop experiments.


Future plans

70

For python users:

Integrate CNMF-E into an open-source package CaImAn (Calcium Imaging Analysis )

https://github.com/simonsfoundation/CaImAn

https://github.com/simonsfoundation/CaImAn


Acknowledgements

71

@CMU@Columbia

@Simons Foundation

@UNC

@UCSF

@Harvard


Acknowledgements

72

All CNMF-E users

https://beat-ica.slack.com/

Date post:	31-Jan-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Computational Tutorial: Calcium Imaging Data Cell Extraction · 2017. 11. 27. · Computational...

Documents