Abstract Understanding causal relationships, oreffective connectivity, between parts of the brain is ofutmost importance because a large part of the brain’sactivity is thought to be internally generated and,hence, quantifying stimulus response relationshipsalone does not fully describe brain dynamics. Pastefforts to determine effective connectivity mostly reliedon model based approaches such as Granger causalityor dynamic causal modeling. Transfer entropy (TE) isan alternative measure of effective connectivity basedon information theory. TE does not require a modelof the interaction and is inherently non-linear. Weinvestigated the applicability of TE as a metric in a testfor effective connectivity to electrophysiological data
Action Editor: Aurel A. Lazar
R. Vicente, M. Wibral, and M. Lindner contributed equally.
ML was funded by the Hessian initiative for thedevelopment of scientific and economic excellence(LOEWE). RV and GP were in part supportedby the Hertie Foundation and the EU (EUproject GABA—FP6-2005-NEST-Path-043309).
R. Vicente · G. PipaMax Planck Institute for Brain Research, Frankfurt,Germany
based on simulations and magnetoencephalography(MEG) recordings in a simple motor task. In particular,we demonstrate that TE improved the detectability ofeffective connectivity for non-linear interactions, andfor sensor level MEG signals where linear methodsare hampered by signal-cross-talk due to volumeconduction.
Keywords Information theory · Effectiveconnectivity · Causality · Information transfer ·Electroencephalography · Magnetoencephalography
1 Introduction
Science is about making predictions. To this aim sci-entists construct a theory of causal relationships be-tween two observations. In neuroscience, one of theobservations can often be manipulated at will, i.e. astimulus in an experiment, and the second observationis measured, i.e. neuronal activity. If we can correctlypredict the behavior of the second observation we haveidentified a causal relationship between stimulus and
M. Wibral (B)MEG Unit, Brain Imaging Center,Goethe University, Frankfurt, Germanye-mail: [email protected]
M. LindnerDepartment of Educational Psychology, Goethe University,Frankfurt, Germany
M. LindnerCenter for Individual Development and Adaptive Educationof Children at Risk (IDeA), Frankfurt, Germanye-mail: [email protected]
46 J Comput Neurosci (2011) 30:45–67
response. However, identifying causal relationships be-tween stimuli and responses covers only part of neu-ronal dynamics—a large part of the brain’s activity isinternally generated and contributes to the responsevariability that is observed despite constant stimuli(Arieli et al. 1996). For the case of internally generateddynamics it is rather difficult to infer a physical causal-ity because a deliberate manipulation of this aspectof the system is extremely difficult. Nevertheless, wecan try to make predictions based on the concept ofcausality as it was introduced by Wiener (1956). InWiener’s definition an improvement of the predictionof the future of a time series X by the incorporationof information from the past of a second time series Yis seen as an indication of a causal interaction from Yto X. Such causal interactions across brain structuresare also called ‘effective connectivty’ (Friston 1994)and they are thought to reveal the information flowassociated to neuronal processing much more preciselythan functional connectivity, which only reflects thestatistical covariation of signals as typically revealed bycross-correlograms or coherency measures. Therefore,we must identify causal relationships between parts ofthe brain, be they single cells, cortical columns, or brainareas.
Various measures of causal relationships, or effectiveconnectivity, exist. They can be divided into two largeclasses: those that quantify effective connectivity basedon the abstract concept of information of random vari-ables (e.g. Schreiber 2000), and those based on specificmodels of the processes generating the data. Meth-ods in the latter class are most widely used to studyeffective connectivity in neuroscience, with Grangercausality (GC, Granger 1969) and dynamic causal mod-eling (DCM, Friston et al. 2003) arguably being mostpopular. In the next two paragraphs we give a shortoverview over the data generation models in GC andDCM and their specific consequences so that the readercan appreciate the fundamental differences betweenthese model based approaches and the informationtheoretic approach presented below:
Standard implementations of GC use a linear sto-chastic model for the intrinsic dynamics of the signaland a linear interaction.1 Therefore, GC is only wellapplicable when three prerequisites are met: (a) Theinteraction between the two units under observationhas to be well approximated by a linear description, (b)the data have to have relatively low noise levels (see
1Historically, however, GC was formulated without explicit as-sumptions about the linearity of the system (Granger 1969) andwas therefore closely related to Wiener’s formal definition ofcausality Wiener (1956).
e.g. Nalatore et al. 2007), and (c) cross-talk betweenthe measurements of the two signals of interest has tobe low (Nolte et al. 2008). Frequency domain variantsof GC such as the partial directed coherence or thedirected transfer function fall in the same category(Pereda et al. 2005).
DCM assumes a bilinear state space model (BSSM).Thus, DCM covers non-linear interactions—at leastpartially. DCM requires knowledge about the input tothe system, because this input is modeled as modu-lating the interactions between the parts of the sys-tem (Friston et al. 2003). DCM also requires a certainamount of a priori knowledge about the network ofconnectivities under investigation, because ultimatelyDCM compares the evidence for several competing apriori models with respect to the observed data. This apriori knowledge on the input to the system and on thepotential connectivity may not always be available, e.g.in studies of the resting-state. Therefore, DCM may notbe optimal for exploratory analyses.
Based on the merits and problems of the methodsdescribed in the last paragraph we may formulate fourrequirements that a new measure of effective connec-tivity must meet to be a useful addition to alreadyestablished methods:
1. It should not require the a priori definition of thetype of interaction, so that it is useful as a tool forexploratory investigations.
2. It should be able to detect frequently observedtypes of purely non-linear interactions. This is be-cause strong non-linearities are observed acrossall levels of brain function, from the all-or nonemechanism of action potential generation in neu-rons to non-linear psychometric functions, such asthe power-law relationship in Weber’s law or theinverted-U relationship between arousal levels andresponse speeds described in the Yerkes-Dodsonlaw (Yerkes and Dodson 1908).
3. It should detect effective connectivity even if therethere is a wide distribution of interaction delaysbetween the two signals, because signaling betweenbrain areas may involve multiple pathways or trans-mission over various axons that connect two areasand that vary in their conduction delays (Swadlowand Waxman 1975; Swadlow et al. 1978).
4. It should be robust against linear cross-talk be-tween signals. This is important for the analysis ofdata recorded with electro- or magnetoencephalog-raphy, that provide a large part of the availableelectrophysiological data today.
The fact that a potential new method should be asmodel free as possible naturally leads to the applica-
J Comput Neurosci (2011) 30:45–67 47
tion of information theoretic techniques. Informationtheory (IT) sets a powerful framework for the quan-tification of information and communication (Shannon1948). It is not surprising then that information the-ory also provides an ideal basis to precisely formulatecausal hypotheses. In the next paragraph, we presentthe connection between the quantification of informa-tion and communication and Wiener’s definition ofcausal interactions (Wiener 1956) in more detail be-cause of its importance for the justification of using ITmethods in this work.
In the context of information theory, the key mea-sure of information of a discrete2 random variable isits Shannon entropy (Shannon 1948; Reza 1994). Thisentropy quantifies the reduction of uncertainty obtainedwhen one actually measures the value of the variable.On the other hand, Wiener’s definition of causal de-pendencies rests on an increase of prediction power. Inparticular, a signal X is said to cause a signal Y whenthe future of signal Y is better predicted by addingknowledge from the past and present of signal X thanby using the present and past of Y alone (Wiener 1956).Therefore, if prediction enhancement can be associatedto uncertainty reduction, it is expected that a causalitymeasure would be naturally expressible in terms ofinformation theoretic concepts.
First attempts to obtain model-free measures of therelationship between two random variables were basedon mutual information (MI). MI quantifies the amountof information that can be obtained about a randomvariable by observing another. MI is based on prob-ability distributions and is sensitive to second and allhigher order correlations. Therefore, it does not relyon any specific model of the data. However, MI sayslittle about causal relationships, because of its lack ofdirectional and dynamical information: First, MI is sym-metric under the exchange of signals. Thus, it cannotdistinguish driver and response systems. And second,standard MI captures the amount of information that isshared by two signals. In contrast, a causal dependenceis related to the information being exchanged ratherthan shared (for instance, due to a common drive ofboth signals by an external, third source). To obtain
2For a continuous random variable the natural generalization ofShannon entropy is its differential entropy. Although differentialentropy does not inherit the properties of Shannon entropy asan information measure, the derived measures of mutual infor-mation and transfer entropy retain the properties and meaningthey have in the discrete variable case. We refer the reader toKaiser and Schreiber (2002) for a more detailed discussion of TEfor continuous variables. In addition, measurements of physicalsystems typically come as discrete random variables because ofthe binning inherent in the digital processing of the data.
an asymmetric measure, delayed mutual information,i.e. MI between one of the signals and a lagged versionof another has been proposed. Delayed MI results inan asymmetric measure and contains certain dynamicalstructure due to the time lag incorporated. Neverthe-less, delayed mutual information has been pointed outto contain certain flaws such as problems due to acommon history or shared information from a commoninput (Schreiber 2000).
A rigorous derivation of a Wiener causal measurewithin the information theoretic framework was pub-lished by Schreiber under the name of transfer entropy(Schreiber 2000). Assuming that the two time seriesof interest X = xt and Y = yt can be approximated byMarkov processes, Schreiber proposed as a measure ofcausality to compute the deviation from the followinggeneralized Markov condition
p(yt+1|ynt , xm
t ) = p(yt+1|ynt ) , (1)
where xmt = (xt, ..., xt−m+1), yn
t = (yt, ..., yt−n+1), whilem and n are the orders (memory) of the Markovprocesses X and Y, respectively. Notice that Eq. (1)is fully satisfied when the transition probabilities ordynamics of Y is independent of the past of X, this isin the absence of causality from X to Y. To measurethe departure from this condition (i.e. the presenceof causality), Schreiber uses the expected Kullback-Leibler divergence between the two probability distri-butions at each side of Eq. (1) to define the transferentropy from X to Y as
T E (X → Y)
=∑
yt+1,ynt ,xm
t
p(yt+1, ynt , xm
t ) log(
p(yt+1|ynt , xm
t )
p(yt+1|ynt )
), (2)
Transfer entropy naturally incorporates directionaland dynamical information, because it is inherentlyasymmetric and based on transition probabilities. In-terestingly, Paluš has shown that transfer entropy canbe rewritten as a conditional mutual information (Paluš2001; Hlavackova-Schindler et al. 2007).
The main convenience of such an information the-oretic functional designed to detect causality is that,in principle, it does not assume any particular modelfor the interaction between the two systems of interest,as requested above. Thus, the sensitivity of transferentropy to all order correlations becomes an advan-tage for exploratory analyses over GC or other modelbased approaches. This is particularly relevant whenthe detection of some unknown non-linear interactionsis required.
48 J Comput Neurosci (2011) 30:45–67
Here, we demonstrate that transfer entropy does in-deed fulfill the above requirements 1–4 and is thereforea useful addition to the available methods for the quan-tification of effective connectivity, when used as a met-ric in a suitable permutation test for independence. Wedemonstrate its ability to detect purely non-linear in-teractions, its ability to deal with a range of interactiondelays, and its robustness against linear cross-talk onsimulated data. This latter point is of particular interestfor non-invasive human electrophysiology using EEGor MEG. The robustness of TE against linear cross-talk in the presence of noise, has to our knowledge notbeen investigated before. We test transfer entropy on avariety of simulated signals with different signal gener-ation dynamics, including biologically plausible signalswith spectra close to 1/f. We also investigate a range oflinear and purely non-linear coupling mechanisms. Inaddition, we demonstrate that transfer entropy workswithout specifying a signal model, i.e. that requirement1 is fulfilled. We extend earlier work (Hinrichs et al.2008; Chávez et al. 2003; Gourvitch and Eggermont2007) by explicitly demonstrating the applicability oftransfer entropy for the case of linearly mixed signals.
2 Methods
The method section is organized in four main parts. Inthe first part we describe how to compute TE numeri-cally. As several estimation techniques could be appliedfor this purpose we quickly review these possibilitiesand give the rationale for our particular choice of es-timator. In the second part, we describe two particu-lar problems that arise in neuroscience applications—delayed interactions, and observation of the signals ofinterest by measurements that only represent linearmixtures of these signals. The third part provides detailson the simulation of test cases for the detection ofeffective connectivity via TE. The last part containsdetails of the MEG recordings in a self-paced finger-lifting task that we chose as a proof-of-concept for theanalysis of neuroscience data.
2.1 Computation of transfer entropy
Transfer entropy for two observed time series xt and yt
can be written as
T E (X → Y)
=∑
yt+u,ydyt ,xdx
t
p(
yt+u, ydyt , xdx
t
)log
p(
yt+u|ydyt , xdx
t
)
p(
yt+u|ydyt
) , (3)
where t is a discrete valued time-index and u denotesthe prediction time, a discrete valued time-interval. ydy
t
and xdxt are dx- and dy-dimensional delay vectors as
detailed below. An estimator of the transfer entropycan be obtained via different approaches (Hlavackova-Schindler et al. 2007). As with other information-theoretic functionals, any estimate shows biases andstatistical errors which depend on the method used andthe characteristics of the data (Hlavackova-Schindleret al. 2007; Kraskov et al. 2004). In some applicationsthe magnitude of such errors is so large that it preventsany meaningful interpretation of the measure. To ourpurposes, it is crucial then to use a proper estimator thatis as accurate as possible under the specific and severeconstraints that most neuronal data-sets present and tocomplement it with an appropriate statistical test. Inparticular, a quantifier of transfer entropy apt for neu-roscience applications should cope with at least threedifficulties. First, the estimator should be robust tomoderate levels of noise. Second, the estimator shouldrely only on a very limited number of data samples. Thispoint is particularly restrictive since relevant neuronaldynamics typically unfolds over just a few hundred ofmilliseconds. And third, due to the need to reconstructthe state space from the observed signals, the estimatorshould be reliable when dealing with high-dimensionalspaces. Under such restrictive conditions, to obtain ahighly accurate estimator of TE is probably impossiblewithout strong modelling assumptions. Unfortunately,strong modelling assumptions require specific informa-tion which is typically not available for neurosciencedata. Nevertheless, some very general and biophysi-cally motivated assumptions are available that enablethe use of particular kernel-based estimators (Victor2002). Here, we build on this framework to derivea data-efficient estimator, detailed below. Even usingthis improved estimator inaccuracies in estimation areunavoidable, specially for the restrictive conditionscommented above, and it is necessary to evaluate thestatistical significance of the TE measures, i.e. we useTE as a statistic measuring dependency of two time se-ries and test against the null hypothesis of independenttime series. Since no parametric distribution of errorsis known for TE, one needs suitable surrogate datato test the null hypothesis of independent time series(‘absence of causality’). Suitable in this context meansthat the surrogate data should be prepared such thatthe causal dependency of interest is destroyed by con-structing the surrogates but trivial dependencies of nointerest are preserved. It is the particular combinationof a data efficient estimator and a suitable statistical testthat forms the core part of this study and its contribu-tion to the field of effective connectivity analysis.
J Comput Neurosci (2011) 30:45–67 49
In the next subsection we detail both, how to obtainan data-efficient estimation of Eq. (3) from the rawsignals, and a statistical significance analysis based onsurrogate data.
2.1.1 Reconstructing the state space
Experimental recordings can only access a limited num-ber of variables which are more or less related to thefull state of the system of interest. However, sensiblecausality hypotheses are formulated in terms of theunderlying systems rather than on the signals beingactually measured. To partially overcome this prob-lem several techniques are available to approximatelyreconstruct the full state space of a dynamical sys-tem from a single series of observations (Kantz andSchreiber 1997).
In this work, we use a Takens delay embedding(Takens 1981) to map our scalar time series into tra-jectories in a state space of possibly high dimension.The mapping uses delay-coordinates to create a setof vectors or points in a higher dimensional space ac-cording to
xdt =(x (t) , x (t−τ) , x (t−2τ) , ..., x (t−(d−1) τ )) . (4)
This procedure depends on two parameters, the di-mension d and the delay τ of the embedding. Whilethere is an extensive literature on how to choosesuch parameters, the different methods proposed arefar away from reaching any consensus (Kantz andSchreiber 1997). A popular option is to take the delayembedding τ as the auto-correlation decay time (act)of the signal or the first minimum (if any) of the auto-information. To determine the embedding dimension,the Cao criterion offers an algorithm based on falseneighbors computation (Cao 1997). However, alter-natives for non-deterministic time-series are available(Ragwitz and Kantz 2002).
The parameters d and τ considerably affect the out-come of the TE estimates. For instance, a low valueof d can be insufficient to unfold the state space ofa system and consequently degrade the meaning ofany TE measure, as will be demonstrated below. Onthe other hand, a too large dimensionality makes theestimators less accurate for a given data length and sig-nificantly enlarges the computing time. Consequently,while we have used the recipes described above toorient our search for good embedding parameters, wehave systematically scanned d and τ to optimize theperformance of TE measures.
2.1.2 Estimating the transfer entropy
After having reconstructed the state spaces of any pairof time series, we are now in a position to estimatethe transfer entropy between their underlying systems.We proceed by first rewriting Eq. (3) as sum of fourShannon entropies according to
T E (X → Y) = S(
ydyt , xdx
t
)− S
(yt+u, ydy
t , xdxt
)
+ S(
yt+u, ydyt
)− S
(ydy
t
). (5)
Thus, the problem amounts to computing thedifferent joint and marginal probability distributionsimplicated in Eq. (5). In principle, there are many waysto estimate such probabilities and their performancestrongly depends on the characteristics of the data tobe analyzed. See Hlavackova-Schindler et al. (2007) fora detailed review of techniques. For discrete processes,the probabilities involved can be easily determined bythe frequencies of visitation of different states. Forcontinuous processes, the case of main interest in thisstudy, a reliable estimation of the probability densitiesis much more delicate since a continuous density hasto be approximated from a finite number of samples.Moreover, the solution of coarse-graining a continuoussignal into discrete states is hard to interpret unlessthe measure converges when reducing the coarseningscale. In the following, we reason for our choice of theestimator and describe its functioning.
A possible strategy for the design of an estimatorrelies on finding the parameters that best fit the sam-ple probability densities into some known distribution.While computationally straightforward such approachamounts to assuming a certain model for the proba-bility distribution which without further constraints isdifficult to justify. From the nonparametric approaches,fixed and adaptive histogram or partition methodsare very popular and widely used. However, othernonparametric techniques such as kernel or nearest-neighbor estimators have been shown to be more dataefficient and accurate while avoiding certain arbitrari-ness stemming from binning (Victor 2002; Kaiser andSchreiber 2002). In this work we shall use an estimatorof the nearest-neighbor class.
Nearest-neighbor techniques estimate smooth prob-ability densities from the distribution of distances ofeach sample point to its k-th nearest neighbor. Conse-quently, this procedure results in an adaptive resolutionsince the distance scale used changes according to theunderlying density. Kozachenko-Leonenko (KL) is anexample of such a class of estimators and a standardalgorithm to compute Shannon entropy (Kozachenko
50 J Comput Neurosci (2011) 30:45–67
and Leonenko 1987). Nevertheless, a naive approachof estimating TE via computing each term of Eq. (5)from a KL estimator is inadequate. To see why, it isimportant to notice that the probability densities in-volved in computing TE or MI can be of very differentdimensionality (from 1 + dx up to 1 + dx + dy for thecase of TE). For a fixed k, this means that different dis-tance scales are effectively used for spaces of differentdimension. Consequently, the biases of each Shannonentropy arising from the non-uniformity of the distrib-ution will depend on the dimensionality of the space,and therefore, will not cancel each other.
To overcome such problems in mutual informationestimates, Kraskov, Stögbauer, and Grassberger haveproposed a new approach (Kraskov et al. 2004). Thekey idea is to use a fixed mass (k) only in the higherdimensional space and project the distance scale setby this mass into the lower dimensional spaces. Thus,the procedure designed for mutual information sug-gests to first determine the distances to k-th nearestneighbors in the joint space. Then, an estimator of MIcan be obtained by counting the number of neighborsthat fall within such distances for each point in themarginal space. The estimator of MI based on thismethod displays many good statistical properties, itgreatly reduces the bias obtained with individual KLestimates, and it seems to become an exact estimatorin the case of independent variables. For these reasons,in this work we have followed a similar scheme toprovide an data-efficient sample estimate for transferentropy (Gomez-Herrero et al. 2010). Thus, we haveobtained an estimator that permits us, at least partially,to tackle some of the main difficulties faced in neuronaldata sets mentioned in the beginning of the Methodssection. In summary, since the estimator is more dataefficient and accurate than other techniques (especiallythose based on binning), it allows to analyze shorterdata sets possibly contaminated by small levels of noise.At the same time, the method is especially geared tohandle the biases of high dimensional spaces naturallyoccurring after the embedding of raw signals.
As to computing time, this class of methods spendsmost of resources in finding neighbors. It is then highlyadvisable to implement an efficient search algorithmwhich is optimal for the length and dimensionality ofthe data to be analyzed (Cormen et al. 2001). For thecurrent investigation, the algorithm was implementedwith the help of OpenTSTool (Version1.2 on Linux64 bit; Merkwirth et al. 2009). The full set of methodsapplied here is available as an open source MATLABtoolbox (Lindner et al. 2009).
In practice, it is important to consider that this kernelestimation method carries two parameters. One is the
mass of the nearest-neighbors search (k) which controlsthe level of bias and statistical error of the estimate.For the remainder of this manuscript this parameterwas set to k = 4, as suggested in Kraskov et al. (2004),unless stated otherwise. The second parameter refersto the Theiler correction which aims to exclude au-tocorrelation effects from the density estimation. Itconsists of discarding for the nearest-neighbor searchthose samples which are closer in time to a referencepoint than a given lapse (T). Here, we chose T = 1 act,unless stated otherwise. In general, it means that eventhough TE does not assume any particular model, itsnumerical estimation relies on at least five different pa-rameters; the embedding delay (τ ) and dimension (d),the mass of the nearest neighbor search (k), the Theilercorrection window (T), and the prediction time (u).The latter accounts for non-instantaneous interactions.Specifically it reflects that in that case an increment ofpredictability of one signal thanks to the incorporationof the past of others should only occur for a certainlatency or prediction time. Since axonal conductiondelays among remote areas can amount to tens ofmilliseconds (Swadlow and Waxman 1975; Swadlow1994), its incorporation for a sensible causality analysisof neuronal data sets is important for the results as weshall see below.
2.1.3 Signif icance analysis
To test the statistical significance of a value for TEobtained we used surrogate data. In general, generatingsurrogate data with the same statistical properties asthe original data but selectively destroying any causalinteraction is difficult. However, when the data set hasa trial structure it is possible to reason that shufflingtrials generates suitable surrogate data sets for theabsence of causality hypothesis if stationarity and trialindependency are assured. On these data we have thenused a permutation test (∼19,000 permutations) onthe unshuffled and shuffled trials to obtain a p-value.P-values below 0.05 were considered significant. Wherenecessary a correction of this threshold for multiplecomparisons was applied using the false discovery rate(FDR, q < 0.05; Genovese et al. 2002).
2.2 Particular problems in neuroscience data:instantaneous mixing and delayed interactions
Neuroscience data have specific characteristics thatchallenge a simple analysis of effective connectivity.First, the interaction may involve large time delays ofunknown duration and, second, the data generated bythe original processes may not be available but only
J Comput Neurosci (2011) 30:45–67 51
measurements that represent linear mixtures of theoriginal data—as is the case in EEG and MEG. In thissection we describe a number of additional tests thatmay help to interpret the results obtained by computingTE values from these types of neuroscience data.
Tests for instantaneous linear mixing and for multiplenoisy observations of a single source Instantaneous,linear mixing of the original signals by the measure-ment process as is always present in MEG and EEGdata. This may result in two problems: First, linearmixing may reduce signal asymmetry and, thus, makeit more difficult to detect effective connectivity of theunderlying sources. This problem is mainly one ofreduced sensitivity of the method and maybe dealtwith, e.g. by increasing the amount of data. A secondproblem arises when a single source signal with aninternal memory structure is observed multiple timeson different channels with individual channel noise. Asdemonstrated before (Nolte et al. 2008) this latter casecan result in false positive detection of effective con-nectivity for methods based on Wiener’s definition ofcausality (Wiener 1956). This problem is more severe,because it reduces the specificity of the method. Asan example of this problem think of an AR process oforder m, s(t)
s(t) =m∑
i=1
αis(t − i) + ηs(t) (6)
that is mixed with a mixing parameter ε onto two sensorsignals X ′, Y ′ in the following way
X ′(t) = s(t) , (7)
Y ′(t) = (1 − ε)s(t) + εηY , (8)
where the dynamics for Y ′ can be rewritten as
Y ′(t) = (1 − ε)
m∑
i=1
αi X ′(t − i) + (1 − ε)ηs + εηY . (9)
In this case TE will identify a causal relationship be-tween X ′ and Y ′ as it detects the relationship betweenthe past of X ′ and the present X ′ that is contained in Y ′as (1 − ε)ηs. Therefore, we implemented the followingadditional test (‘time-shift test’) to avoid false positivereports for the case of instantaneous, linear mixing:We shifted the time series for X ′ by one sample intothe past X ′′(t) ←↩ X ′(t + 1) such that a potential in-stantaneous mixing becomes lagged and thereby causalin Wiener’s sense. For instantaneous mixing processesTE values increase for the interaction from the shiftedtime series X ′′(t) to Y ′ compared to the interactionfrom the original time series X ′(t) to Y ′. Therefore,
an increase of this kind may indicate the presence ofinstantaneous mixing. The actual shift test implementsthe null hypothesis of instantaneous mixing and thealternative hypothesis of no instantaneous mixing in thefollowing way:
H0 : T E(X ′′(t) → Y ′) ≥ T E(X ′(t) → Y ′)
H1 : T E(X ′′(t) → Y ′) < T E(X ′(t) → Y ′) (10)
If the null hypothesis of instananeous mixing is notdiscarded by this test, i.e. if TE values for the originaldata are not significantly larger than those for theshifted data, then we have to discard the hypothesis ofa causal interaction from X ′ to Y ′. Therefore, whendata potentially contained instantaneous mixing, wetested for the presence of instantaneous mixing beforeproceeding to test the hypothesis of effective connec-tivity. More specifically, this test was applied for theinstantaneously mixed simulation data (Figs. 4, 5, 6)and the MEG data (Fig. 8). In general, we suggest to usethis test, whenever the data in question may have beenobtained via a measurement function that containedlinear, instantaneuos mixing.
A less conservative approach to the same problemwould be to discard data for TE analysis only when wehave significant evidence for the presence of instanta-neous mixing. In this case the hypotheses would be:
H0 : T E(X ′′(t) → Y ′) ≤ T E(X ′(t) → Y ′)
H1 : T E(X ′′(t) → Y ′) > T E(X ′(t) → Y ′) (11)
In this case we would proceed analysing the data ifwe did not have to reject H0. For the remainder ofthis manuscript, however, we stick to testing the moreconservative null hypothesis presented in Eq. (10).
Delayed interactions, Wiener’s def inition of causality,and choice of embedding parameters This paragraphintroduces a difficulty related to Wiener’s definitionof causality. As described above, non-zero TE valuescan be directly translated into improved predictions inWiener’s sense by interpreting the terms in Eq. (2) astransition probabilities, i.e. as information that is usefulfor prediction. TE quantifies the gain in our knowledgeabout the transition probabilities in one system Y, thatwe obtain if we condition these probabilities on thepast values of another system X. It is obvious that thisgain, i.e. the value of TE, can be erroneously high,if the transition probabilities for system Y alone arenot evaluated correctly. We now describe a case wherethis error is particularly likely to occur: Consider twoprocesses with lagged interactions and long autocorre-lation times. We assume that system X drives Y with an
52 J Comput Neurosci (2011) 30:45–67
interaction delay δ (Fig. 1). A problem arises if we testfor a causal interaction from Y to X, i.e. the reversedirection compared to the actual coupling, and do nottake enough care to fully capture the dynamics of X viaembedding. If for example the embedding dimension dor the embedding delay τ was chosen too small, thensome information contained in the past of X is notused although it would improve (auto-) prediction. Thisinformation is actually transferred to Y via the delayedinteraction from X to Y. It is available in Y with a delayδ, and therefore, at time-points were data from Y isused for the prediction of X. As stated before this infor-mation is useful for the prediction of X. Thus, inclusionof Y will improve prediction. Hence, TE values will be
non-zero and we will wrongly conclude that process Ydrives process X.
2.3 Simulated data
We used simulated data to test the ability of TE touncover causal relations under different situations rel-evant to neuroscience applications. In particular, we al-ways considered two interacting systems and simulateddifferent internal dynamics (autoregressive and 1/ fcharacteristics), effective connectivity (linear, thresh-old and quadratic coupling), and interaction delays (sin-gle delay and a distribution of delays). In addition, wesimulated linear instantaneous mixing processes during
Y(target)
X(source)
r(X
(u),
X(u
-t))
0
delay δ
uτ τ
dused = 3
Y X ?
Fig. 1 Illustration of false positive effective connectivity due toinsufficient embedding for delayed interactions. Source signalX drives target signal Y with a delay δ. The internal memoryof process X is reflected in the slowly decaying autocorrelationfunction (top). For the evaluation of TE from Y to X, X isembedded for auto-prediction with d = 3 and τ , as indicated bythe dark gray box. The data point of X that is to be predictedwith prediction time u is indicated by the star shaped symbol.Data points used for auto-prediction are indicated by f illed circleson signal X. Data points used for cross-prediction from Y to Xare indicated by f illed circles on signal Y. Due to the delayed
interaction from X to Y information about X earlier than theembedding time gets transferred from X to Y where it getsincluded in the embedding (open circle). Y contains informationthe history of X that is useful for predicting X (see open circle,autocorrelation curve) but not contained in the embedding usedon X. Hence, inclusion of Y will improve the prediction of X andfalse positive effective connectivity is found. Introducing a largerembedding dimension or or larger embedding delay, incorporatesthis information into the embedding of X. Examples of this effectcan be found in Tables 1 and 2
J Comput Neurosci (2011) 30:45–67 53
measurement, because of their relevance for EEG andMEG.
2.3.1 Internal signal dynamics
We have simulated two types of complex internal signaldynamics. In the first case, an autoregressive processof order 10, AR(10), is generated for each system. Thedynamics is then given by
x(t + 1) =9∑
i=0
αix(t − i) + ση(t) , (12)
where the coefficients αi are drawn from a normalizedGaussian distribution, the innovation term η representsa Gaussian white noise source, and σ controls therelative strength of the noise contribution. Notice, thatwe use here the typical notation in dynamical systemswhere the innovation term η(t) is delayed one unit withrespect the output x(t + 1).
As a second case, we have considered signals with a1/ f θ profile in their power spectra. To produce such sig-nals we have followed the approach in Granger (1980).Accordingly, the 1/ f θ time series are generated as theaggregation of numerous AR(1) processes with an ap-propriate distribution of coefficients. Mathematically,each 1/ f θ signal is then given by
x(t + 1) = 1N
N∑
i=1
ri(t) , (13)
where we aggregate over N = 500 AR(1) processeseach described as
ri(t) = αiri(t − 1) + ση(t) , (14)
with the coefficients αi randomly chosen according tothe probability density function ∼ (1 − α)1−θ .
2.3.2 Types of interaction
To simulate a causal interaction between two systemswe added to the internal dynamics of one process (Y)a term related to the past dynamics of the other (X).Three types of interaction or effective connectivitywere considered; linear, quadratic, and threshold. Inthe linear case, the interaction is proportional to theamount of signal at X. The last two cases representstrong non-linearities which challenge approaches ofdetection based on linear or parametric methods. Theeffective connectivity mediated by the threshold func-tion is of special relevance in neuroscience applicationsdue to the approximated all or none character of theneuronal spike generation and transmission. Mathe-
matically, the update of y(t) is then modeled by the ad-dition of an interaction term such that the full dynamicsis described as
y(t) = D(y−) +
⎧⎪⎨
⎪⎩
γlinx(t − δ) if linear,
γquadx2(t − δ) if quadratic,
γthresh1
1+exp(b 1+b 2x(t−δ))if threshold,
where D(.) represents the internal dynamics (AR(10)or 1/ f ) of y and y− represents past values of y. Inthe last case, the threshold function is implementedthrough a sigmoidal with parameters b 1 and b 2 whichcontrol the threshold level and its slope, respectively.Here, b 1 was set to 0 and b 2 was set to 50. In all cases,δ represents a delay which typically arises from thefinite speed of propagation of any influence betweenphysically separated systems. Note that since we dealwith discrete time models (maps) in our modeling δ
takes only positive integer values.In case that two systems interact via multiple path-
ways it is possible that different latencies arise intheir communication. For example, it is known thatthe different characteristics of the axons joining twobrain areas typically lead to a distribution of axonalconduction delays (Swadlow et al. 1978; Swadlow 1985).To account for that scenario we have also simulated thecase where δ instead of a single value is a distribution.Accordingly, for each type of interaction we have con-sidered the case where the interaction term is
Interaction term
=
⎧⎪⎨
⎪⎩
∑δ′ γlinx(t − δ′) if linear,
∑δ′ γquadx2(t − δ′) if quadratic,
∑δ′ γthresh
11+exp(b 1+b 2x(t−δ′)) if threshold ,
where the sums are extended over a certain domainof positive integer values. In the results section weconsider the case in which δ′ takes values on a uniformdistribution of width 6 centered around a given delay.
The coupling constants γlin, γquad, γthresh were al-ways chosen such that the variance of the interactionterm was comparable to the variance of y(t) that wouldbe obtained in the absence of any coupling.
2.3.3 Linear mixing
Linear instantaneous mixing is present in human non-invasive electrophysiological measurements such asEEG or MEG and has been shown to be problem-atic for GC (Nolte et al. 2008). The problem we en-counter for linearly and instantaneously mixed signalsis twofold: On the one hand, instantaneous mixing fromcoupled source signals onto sensor signals by the mea-
54 J Comput Neurosci (2011) 30:45–67
surement process degrades signal asymmetry (Tognoliand Scott Kelso 2009), it will therefore be harder todetect effective connectivity. On the other hand—asshown in Nolte et al. (2008)—instantaneous presence ofa single source signal in two measurements of differentsignal to noise ratio may be interpreted as effectiveconnectivity erroneously. To test the influence of linearinstantaneous mixing we created two test cases:
(A) The first test case consisted in unidirectionallycoupled signal pairs X → Y generated from cou-pled AR(10) processes as described above andthen transformed into two linear instantaneousmixtures Xε, Yε in the following way:
Xε(t) = (1 − ε)X(t) + εY(t) (15)
Yε(t) = εX(t) + (1 − ε)Y(t) (16)
Here, ε is a parameter that describes the amountof linear mixing or ‘signal cross-talk’. A valueof ε of 0.5 means that the mixing leads to twoidentical signals and, hence, no significant TEshould be observed. We then investigated forthree different values of ε = (0.1, 0.25, 0.4) howwell TE detects the underlying effective connec-tivity from X to Y if only the linear mixturesXε, Yε are available.
(B) The second test case consisted in generating mea-surement signals Xε, Yε in the following way:
Xε(t) = s(t) (17)
Yε(t) = (1 − ε)s(t) + εηY (18)
Here, s(t) is the common source, a mean-freeAR(10) process with unit variance. s(t) is mea-sured twice: once noise free in Xε and oncedampened by a factor (1 − ε) and corrupted byindependent Gaussian noise of unit variance, ηY ,in Yε . Here, we tested the ability of our im-plementation of TE to reject the hypothesis ofeffective connectivity. This second test case is ofparticular importance for the application of TEto EEG and MEG measurements where oftena single source may be observed on two sensorsthat have different noise characteristics, i.e. dueto differences in contact resistance of the EEGelectrodes or the characteristics of the MEG-SQUIDS.
2.3.4 Choice of embedding parameters for delayedinteractions
To demonstrate the effects of suboptimal embeddingparameters for the case of delayed interactions we
simulated processes with autoregressive order 10(AR(10)) dynamics, three different interaction delays(5, 20, 100 samples) and all three coupling types (linear,threshold, quadratic). The two processes were coupledunidirectionally X → Y. 15, 30, 60, and 120 trials weresimulated. We tested for effective connectivity in bothpossible directions using permutation testing. All cou-pled processes were investigated with three differentprediction times u of 6, 21, and 101 samples. Theremaining analysis parameters were: d = 7, τ = 1 act,k = 4, T = 1 act. In addition, we simulated processeswith 1/ f dynamics, an interaction delay δ of 100 sam-ples and a unidirectional, quadratic coupling. 30 trialswere simulated and we tested for effective connectivityin both directions. These coupled processes were inves-tigated with all possible combinations of three differentembedding dimensions d = 4, 7, 10, two differentembedding delays τ = 1 act or τ = 1.5 act and threedifferent prediction times u = 6, 21, 101 samples. Theremaining analysis parameters were: k = 4, T = 1 act.Results are presented in Tables 1 and 2.
2.4 MEG experiment
Rationale In order to demonstrate the applicability ofTE to neuroscience data obtained non-invasively weperformed MEG recordings in a motor task. Our aimwas to show that TE indeed gave the results that wereexpected based on prior, neuroanatomical knowledge.To verify the correctness of results in experimentaldata is difficult because no knowledge about the ulti-mate ground truth exists when data are not simulated.Therefore, we chose an extremely simple experiment—self-paced finger lifting of the index fingers in a self-chosen sequence—where very clear hypotheses aboutthe expected connectivity from the motor cortices tothe finger muscles exist.
Subjects and experimental task Two subjects (S1, m,RH, 38 yrs; S2, f, RH, 23 yrs) participated in theexperiment. Subjects gave written informed consentprior to the recording. Subjects had to lift the right andleft index finger in a self-chosen randomly alternatingsequence with approximately 2s pause between succes-sive finger liftings. Finger movements were detectedusing a photosensor. In addition, an electromyogram(EMG) response was recorded from the extensor mus-cles of the the right and left index fingers.
Recording and preprocessing MEG data wererecorded using a 275 channel whole head system(OMEGA2005, VSM MedTech Ltd., Coquitlam, BC,Canada) in a synthetic 3rd order gradiometer configu-ration. Additional electrocardiographic, -occulographicc
J Comput Neurosci (2011) 30:45–67 55
Table 1 Detection of true and false effective connectivity for afixed embedding dimension d of 7, and an embedding delay τ of1 autocorrelation time
Given is the detected effective connectivity in dependence of theparameter prediction time u for data with different interactiondelays δ of 5, 20, and 100 samples. Data were simulated withautoregressive order ten dynamics and unidirectional couplingX → Y via three different coupling functions (linear, threshold,quadratic). Simulation results based on 120 trials. Note: falsepositives emerge for short interaction delays δ, i.e. the inclusionof more recent samples of X, i.e. samples that are just before theearliest embedding time-point; false positives in these cases aresuppressed using a larger prediction time, i.e. moving the embed-ding of X and the samples of X that are transferred to Y furtherinto the past; short interaction delays can robustly be detectedwith prediction times that are longer than the interaction delay,if the difference is not excessive
and -myographic recordings were made to measurethe electrocardiogram (ECG), horizontal andvertical electrooculography (EOG) traces, and theelectromyogram (EMG) for the extensor muscles ofthe right and left index fingers. Data were hardwarefiltered between 0.5 and 300 Hz and digitized at asampling rate of 1.2 kHz. Data were recorded intwo continuous sessions lasting 600 s each. For theanalysis of effective connectivity between scalp sensorsand the EMG, data were preprocessed using theFieldtrip open-source toolbox for MATLAB (http://
Table 2 Detection of true and false effective connectivity independence of the parameters embedding delay τ , embeddingdimension d, and prediction time u for data with unidirectionalcoupling X → Y via a quadratic function, 1/ f dynamics and aninteraction delay δ of 100 samples
Simulation results based on 30 trials. Note how a larger τ elim-inates false positive TE results for effective connectivity. Alsonote how the delocalization in time provided by the embeddingenables the detection of effective connectivities also for interac-tion delays larger than the prediction time
fieldtrip.fcdonders.nl/; version 2008-12-10). Data weredigitally filtered between 5 and 200 Hz and then cutin trials from −1,000 ms before to 90 ms after thephotosensor indicated a lift of the left or right indexfinger. This latency range ensured that enough EMGactivity was included in the analysis. We used theartifact rejection routines implemented in Fieldtrip todiscard trials contaminated with eye-blinks, muscularactivity and sensor jumps.
Analysis of ef fective connectivity at the MEG sensorlevel using transfer entropy Effective connectivity wasanalyzed using the algorithm to compute transfer en-tropy as described above. The algorithm was imple-mented as a toolbox (Lindner et al. 2009) for Fieldtripdata structures (http://fieldtrip.fcdonders.nl/) in MAT-LAB. The nearest neighbour search routines were im-plemented using OpenTSTool (Version1.2 on Linux 64bit; Merkwirth et al. 2009). Parameters for the analysiswere chosen based on a scanning of the parameterspace, to obtain maximum sensitivity. In more detailwe computed the difference between the transfer en-tropy for the MEG data and the surrogate data for allcombinations of parameters chosen from: τ = 1 act, u ∈[10, 16, 22, 30, 150], d ∈ [4, 5, 7], k ∈ [4, 5, 6, 7, 8, 9, 10].We performed the statistical test for a significant de-viation from independence for each of these parame-tersets. This way a multiple testing problem arose, inaddition to the multiple testing based on the multipledirected intercations between the chosen sensors (seenext paragraph). We therefore performed a correc-
tion for multiple comparisons using the false discov-ery rate (FDR, q < 0.05, Genovese et al. 2002). Theparameter values with optimum sensitivity, i.e. mostsginificant results across sensor pairs after corrcetionfor multiple comparison were: embedding dimensionsd = 7, embedding delay τ = 1 act, forward predictiontime u = 16 ms, number of neighbors considered fordensity estimations k = 4, time window for exclusionof temporally correlated neighbors T = 1act. In addi-tion we required that prediction should be possible forat least 150 samples, i.e. individual trials where thecombination of a long autocorrelation time and theembedding dimension of 7 did not leave enough datafor prediction were discarded. We required that at least30 trials should survive this exclusion step for a datasetto be analyzed.
Even a simple task like self-paced lifting of the left orright index finger potentially involves a very complexnetwork of brain areas related to volition, self-pacedtiming, and motor execution. Not all of the involvedcausal interactions are clearly understood to date. Wetherefore focused on a set on interactions where clear-cut hypothesis about the direction of causal interactionsand the differences between the two conditions existed:We examined TE from the three bilateral sensor pairsdisplaying the largest amplitudes in the magneticallyevoked fields (MEFs) (compare Fig. 7) before onsetof the two movements (left or right finger lift) to bothEMG channels. This also helped to reduce computationtime, as for an all-to-all analysis of effective connectiv-ity at the MEG and EMG sensor level would involvethe analysis of 277 × 276 directed connections. We thentested connectivities in both conditions against eachother by comparing the distributions of TE values in thetwo conditions using a permutation test. For this lattercomparison a clear lateralization effect was expected, astask related causal interactions common to both condi-tions should cancel. Activity in at least three differentfrequency bands has been found in the motor cortexand it has been proposed that each of these differentfrequency bands subserves a different function:
– A slow rhythm (6–10 Hz) has been postulated toprovide a common timing for agonist/antagonistmuscles pairs in slow movements and is thoughtto arise from from synchronization in a cerebello-thalamo-cortical loop (Gross et al. 2002). The cou-pling of cortical (primary motor cortex M1, pri-mary somatosensory cortex S1) activity to muscularactivity was proposed to be bidirectional (Grosset al. 2002) in this frequency range. The couplingmay also depend on oscillations in spinal stretchreflex loops (Erimaki and Christakos 2008).
– Activity in the beta range (∼20 Hz) has been sug-gested to subserve the maintenance of current limbposition (Pogosyan et al. 2009) and strong cortico-muscular coherence in this band has been foundin isometric contraction accordingly (Schoffelen etal. 2008). Coherent activity in the beta band hasalso been demonstrated between bilateral motorcortices (Mima et al. 2000; Murthy and Fetz 1996).
– In contrast, motor-act related activity in the gammaband (>30 Hz) is reported less frequently and its re-lation to motor control is less clearly understood todate (Donoghue et al. 1998). We therefore focusedour analysis on a frequency interval from 5–29 Hz.
Note that we omitted the frequently proposed pre-processing of the EMG traces by rectification (Myerset al. 2003), as TE should be able to detect effectiveconnectivity without this additional step.
3 Results
3.1 Overview
In this section we first present the analysis of effectiveconnectivity in pairs of simulated signals {X,Y}. Allsignal pairs were unidirectionally coupled from X to Y.We used three coupling functions: linear, threshold anda purely non-linear quadratic coupling. We simulatedtwo different signal dynamics, AR(10) processes andprocesses with 1/ f spectra, that were close to spectraobserved in biological signals. The two signals of apair always had similar characteristics. We always ana-lyzed both directions of potential effective connectivity:X → Y and Y → X to quantify both, sensitivity andspecificity of our method.
In addition to this basic simulation we investigatedthe following special cases: coupling via multiple cou-pling delays for linear and threshold interactions, lin-early mixed observation of two coupled signals for lin-ear and threshold coupling, and observation of a singlesignal via two sensors with different noise levels. In thislast case no effective connectivity should be detected.The absence of false positives in this latter case is ofparticular importance for EEG and MEG sensor-levelanalysis.
As a proof of principle we then applied the analy-sis of effective connectivity via TE to MEG signalsrecorded in a self-paced finger lifting task. Here the aimwas to recover the known connectivity from contralat-eral motor cortices to the muscles of the moved limb,via a comparison of effective connectivty for left andright finger lifting.
J Comput Neurosci (2011) 30:45–67 57
3.2 Simulation study
Detection of non-linear interactions for various signaldynamics Transfer entropy in combination with per-mutation testing correctly detected effective connectiv-ity (X → Y) for both, autoregressive order 10 and 1/ fsignal dynamics and all three simulated coupling types(linear, threshold, quadratic) if at least 30 trials wereused to compute statistics (Fig. 2). No false positives,i.e. significant results for the direction Y → X, were
observed. We note that the cross-correlation functionbetween the signals X and Y were flat when couplednon-linearly, which indicates that linear approachesmay be insufficient to detect a significant interaction inthose cases.
Detection of interactions with multiple interaction de-lays The statistical evaluation of TE values robustlydetected the correct direction of effective connectivity(X→Y) for the two unidirectionally coupled AR(10)
Fig. 2 Detection of effectiveconnectivity by TE fortwo unidirectionallycoupled signals (X → Y).(a–c) Signals generatedfrom an autoregressive orderten process and coupled via(a) linear, (b) threshold,and (c) quadratic coupling.(d–f) Signals generatedwith dynamics of a 1/ fnoise process and coupledvia (d) linear, (e) threshold,and (f) quadratic coupling.A single interaction delayof 20 samples was used.Time courses of source(X) and target (Y) signalson the left and results ofpermutation testing for avarying number of trials(15, 30, 60, 120) on the right.Black bars indicate (1-p)values for coupling X → Y(true coupling direction),gray bars indicate values of(1-p) for coupling Y → X.The dashed line correspondsto significant effectiveconnectivity (p < 0.05) X
(source)
process: autoregressive order 10; delay: single delay - 20 samples; couling: X Y (TE-parameters: d=7, τ =act(Y), u=21)
linea
rth
resh
old
quad
ratic
4
-4
0
0 100 200 300
Y(target)
X(source)
Nr of trials
Y
Y
X
X
Y XX Y
1
015 30 60 120
1
01
- p
15 30 60 120
delay = 20
1
015 30 60 120
α=.05(a)
(b)
(c)
samples
z(a.
u.)
process: 1/f noise; delay: single delay - 20 samples; coupling: X Y(TE-parameters: d=7, τ =1.5*act(Y), u=21)
linea
rth
resh
old
quad
ratic
4
-4
0
0 100 200 300
Y(target)
Nr of trials
Y
Y
X
X
Y XX Y
1
015 30 60 120
1
0
1 -
p
15 30 60 120
delay = 20
1
015 30 60 120
α=.05(d)
(e)
(f)
samples
z(a.
u.)
58 J Comput Neurosci (2011) 30:45–67
time series (X,Y), coupled via a range of delays δ from17–23 samples, and for the two unidirectionally coupled1/ f time series, coupled via a range of delays δ from 97-103 samples. The correct coupling direction (X → Y)was found for all three investigated coupling functions(linear, threshold, quadratic), even if only 15 trials wereinvestigated (Fig. 3). For these analysis we used a pre-diction time u of 21 samples for the case of a delay δ of17–23 samples, and a prediction time u of 101 samplesfor the delay δ of 97–103 samples. Correct detection
of effective connectivity was also possible when usinga prediction time u of 21 samples for the delay δ of97–103 samples, i.e. a prediction time that was shorterthan the interaction delay (data not shown). This wasexpected because of the delocalization in time providedfor by the delay embedding. However, no effectiveconnectivity was detected when using a prediction timeu of 101 samples for a interaction delay δ of 17–23samples, i.e. when using a prediction time that wasconsiderably longer than the interaction delay (data not
Fig. 3 Detection of effectiveconnectivity by TE for twounidirectionally coupledtime series (X → Y) witha range of coupling delaysas indicated by the shadedboxes in (a) and (d).(a–c) autoregressive orderten processes; interactiondelays 17–23 samples.(a) Linear interaction,(b) threshold coupling,and (c) quadratic coupling.(d–f) 1/ f processes;interaction delays 97–103samples. (d) Linearinteraction, (e) thresholdcoupling, and (f) quadraticcoupling. Time series areplotted on the left, resultsof permutation testing fordifferent numbers ofsimulated trials (15, 30,60, 120) on the right. Blackbars indicate values of (1-p)for coupling X → Y (truecoupling direction), graybars indicate values of (1-p)for coupling Y → X. Thedashed line correspondsto significant effectiveconnectivity (p < 0.05)
process: autoregressive order 10; multiple delays 17-23 samples; coupling X Y(TE-parameters: d=7, τ=act(Y), u=21)
shown; compare Table 1 for single interaction delays).No false positive effective connectivities (Y→X) werefound. However, relatively high values for (1-p) forsome cases indicate that the embedding parameterswere not optimally chosen, as discussed below.
Detection of ef fective connectivity from linearly mixedmeasurement signals In order to investigate the appli-cation of TE to EEG and MEG sensor signals, wherethe signals from the processes in question can only beobserved after linear mixing processes, we simulatedtwo unidirectionally coupled AR(10) signals (X → Ywith linear or threshold coupling). These signals thenunderwent a symmetric linear mixing process in depen-dence of a parameter ε in the range from 0.1 to 0.4,where a value of ε = 0.5 would indicate identical mixedsignals (see Eqs. (15), (16)). For the case of linearly
coupled source signals TE indicated effective connec-tivity in direction from the sensor signal Xε that had ahigher contribution from the driving process (X) to thesensor Yε dominated by the receiving process (Y) forall investigated cases of linearly mixed measurementsignals except for the case of ε = 0.4. In this case TEdetected the correct direction of the interaction anddid not result in false positive detection, however, thetime-shift test indicated the presence of instantaneousmixing and the result could not be counted as a cor-rect detection of effective connectivity. For the case ofsource signals that were coupled via a threshold func-tion TE in combination with the time-shift test correctlyidentified effective connectivty and did not result infalse positive detection for all of the investigated linearmixing strengths. These observations held even if only15 trials were evaluated (Figs. 4 and 5).
Fig. 4 Simulation resultsfor linearly mixed measure-ments (Xε , Yε ) of twounidirectionally and linearlycoupled underlying sourcesignals (X → Y). (a) Mixingmodel and original auto-regressive source timecourses X, Y. (b–d) Effectiveconnectivity betweensensor-level signals Xε , Yε .Left statistics of permutationtests of TE values for theoriginal sensor level dataagainst trial-shuffledsurrogate data afterapplication of the additionaltime-shift test. The plotscontain values of (1-p) independence of the numberof investigated number oftrials. Black bars indicatevalues for the effectiveconnectivity from the sensordominated by the drivingsource signal (Xε) to thesensor dominated by thereceiving source signal (Yε).Light grey bars indicate thereverse direction of effectiveconnectivity. The dashed linecorresponds to siginificanteffective connectivity(p < 0.05). Right time-courses of signals Xε andYε for a single trial
process: autoregressive order 10; coupling: linear; single delay; observation: linearly mixed(TE-parameters: d=7, τ=1.5*act(Y), u=21)
Y(target)
X(source)
delay = 20
raw
dat
a
X(t)
Xε(t) = (1-ε)X(t) + εY(t)
Yε(t) = (1-ε)Y(t) + εX(t)
Y(t)
(1-ε)(1-ε)
εε
(a)
Yε X
ε
Xε Y
ε
ε =
0.1
1
015 30 60 120
α=.05(b)
Yε
Xε
ε =
0.2
5
Yε
Xε
1
015 30 60 120
(c)
ε =
0.4
4
-4
0
0 100 200 300Nr of trials
Yε
Xε
1
0
1 -
p
15 30 60 120
(d)
samples
z(a.
u.)
60 J Comput Neurosci (2011) 30:45–67
Fig. 5 Simulation resultsfor linearly mixed measure-ments (Xε , Yε ) of twounidirectionally coupledunderlying source signals(X → Y) coupled via athreshold function. (a)Mixing model and originalautoregressive source timecourses X, Y. (b–d) Effectiveconnectivity betweensensor-level signals Xε , Yε .Left statistics of permutationtests of TE values for theoriginal sensor level dataagainst trial-shuffledsurrogate data afterapplication of the additionaltime-shift test. Black barsindicate values for theeffective connectivity fromthe sensor dominated by thedriving source signal (Xε) tothe sensor dominated by thereceiving source signal (Yε).Light grey bars indicate thereverse direction of effectiveconnectivity. The dashed linecorresponds to siginificanteffective connectivity(p < 0.05). Right time-coursesof signals Xε and Yε for asingle trial
ε =
0.1
ε =
0.2
5ε
= 0
.4
4
-4
0
0 100 200 300
Y(target)
X(source)
Nr of trials
Yε
Yε
Xε
Xε
1
015 30 60 120
1
0
1 -
p
15 30 60 120
delay = 20
Yε X
ε
Xε Y
ε
1
015 30 60 120
α=.05(b)
(c)
(d)
samples
z(a.
u.)
raw
dat
a
Yε
Xε
X(t)
Xε(t) = (1-ε)X(t) + εY(t)
Yε(t) = (1-ε)Y(t) + εX(t)
Y(t)
(1-ε)(1-ε)
εε
(a)
process: autoregressive order 10; coupling: thresh. ; single delay; observation: lin. mixed(TE-parameters: d=7, τ=1.5*act(Y), u=21)
Robustness against instantaneous mixing To quantifythe false positive rates when applying transfer entropyto multiple observations of the same signal, but withdifferential noise, we simulated an autoregressive or-der 10 process and two observation of this process:one noise free observation, Xε , and a second obser-vation, Yε , corrupted by a varying amount of whitenoise (Fig. 6(a) and (b)). Similar to the performanceof GC in this case (Nolte et al. 2008), the applicationof TE resulted in a considerable number of false posi-tive detections of effective connectivity from the noisefree sensor signal to the noise-corrupted sensor signal(Fig. 6(c)). However, application of the time-shiftingtest as proposed in the methods section removed allfalse positive cases.
Choosing embedding parameters for delayed interac-tions To demonstrate the importance of correct em-bedding we simulated unidirectionally coupled signalswith various interaction delays and analyzed effectiveconnectivity with different choices for the embeddingdimension d, the embedding delay τ and the predic-tion time u (Tables 1 and 2). As expected because oftheoretical considerations (see Fig. 1), false positiveeffective connectivity is reported for short interactiondelays (5, 20 samples) in combination with short pre-diction times (six samples) and insufficient embedding(d = 4, τ = 1 act). In contrast, if we try to detect longinteractions delays (δ = 20, 100) with too short predic-tion times (u = 6), again with insufficient embedding,the method looses its sensitivity, as expected. This in-
J Comput Neurosci (2011) 30:45–67 61
process: autoregressive order 10; single source; observation: linearly mixed(TE-parameters: d=4, τ=1*act(Y), u=1)
(a)
(d)
(b)
before shift testafter shift test
(c)
0.0 0.25 0.5 0.750
25
50
75
100
fals
e po
sitiv
e ra
te (
%)
ε0.0 0.25 0.5 0.75
0
25
50
75
100
ε
fals
e po
sitiv
e ra
te (
%) Transfer entropy: Y
ε X
εTransfer entropy: Y
ε X
ε
X
ε = 0.0
ε = 0.1
ε = 0.25
ε = 0.4
4
-4
0
0 100 200 300samples
z(a.
u.)
X(t)
Xε(t) = X(t)
Yε(t) = (1-ε)X(t) + εη
1-ε
Yε
Fig. 6 False positive rates for the detection of effective connec-tivity when observing one source via two EEG or MEG sensors.(a) Signal generation by an autoregressive order ten process X(t)and simultaneous observation of this source signal on two sensorsignals Xε , Yε . One of the signals is a copy of the source signal(Xε(t) = X(t)); the other, (Yε), is dampened by a factor of (1 − ε)
and corrupted by white noise εη. (b) Resulting signal time coursesfor the source signal X(t) and the observed sensor signals Yε
for different values of ε. (c) False positive detection rate for
effective connectivity from the noise free sensor signal Xε to thenoise corrupted signal Yε before (dashed line) and after (solidline) the additional time-shift test for instantaneous mixing. Inaccordance with (Nolte et al. 2008) TE without the additional testyields a certain amount of false positive results. (d) False positivedetection rate for effective connectivity from the noise corruptedsignal Yε to the noise free sensor signal Xε . Lines as in (c). Nofalse positives were observed after the additional time shiftingtest
dicates that for given analysis parameters (d,τ ,u) therange of interaction delays δ that can be investigatedreliably is limited (Table 1). The above problem issolved naturally by increasing embedding dimensionsand embedding delays as demonstrated in Table 2—although this may not be possible in practical termssometimes. In our simulations we generally found anembedding delay of τ = 1.5 act in combination withembedding dimensions between 7 and 10 to be moreappropriate than smaller (d = 4, also see Table 2) orlarger embedding dimensions (d = 13, 16, 19, data notshown) or a shorter embedding delay (τ = 1 act). While
it is often proposed to use τ = 1 act for embedding ourdata suggest that for the evaluation of TE it is partic-ularly important to cover most or all of the memoryinherent in both, source and target signals. For our datathis could be be achieved by choosing τ > 1.5 act toprevent against false positive detection of causality inthe presence of delayed interactions. We also observedthat values of the prediction time u close to the actualinteraction delay δ made the analysis of TE both, moresensitive and more robust against false positives, evenfor suboptimal choices of d and τ (Tables 1 and 2).Hence, a choice of u close to δ, e.g. based on prior
62 J Comput Neurosci (2011) 30:45–67
Fig. 7 Neuromagnetic fieldsin a finger lifting task.(a) Single-trial raw tracesof magnetic fields (thin line)measured by two MEGsensors over left (MLT24)and right (MRT24) motorcortex (also compare (d) forthe position of these sensors).In this trial the right fingerwas lifted. (b) Correspondingsingle trial EMG tracesobtained from the left(EMG L) and right (EMG R)forearm. Time ‘0’ indicatesthe sample when the lightbarrier switch detected thefinger lift. (c) Topography ofmagnetic fields averaged overtrials at −50 ms before theregistration of a right indexfinger lift. Note the dipolarpattern over left centralcortex. (d) Layout of theMEG sensors. Sensors usedfor analysis of effectiveconnectivity are indicatedby solid circles. Lines witharrowheads indicate theinvestigated connections
Fig. 8 Differences in effective connectivity (EC) between liftingof the right (RFL) and left index finger (LFL) for subject 1 (left)and subject 2 (right). The investigated frequency band was 5–29Hz encompassing the μ and β rhythms, and avoiding 50 Hz conta-mination. Red lines indicate links where effective connectivity asquantified by TE was significantly larger for lifting of the right
index finger, compared to left. Blue lines indicate links whereeffective connectivity as quantified by TE was significantly largerfor lifting of the left index finger, compared to right. Connectivityfrom contra- and ipsilateral motor cortices to muscles (EMG L,EMG R) of the moved finger is stronger than to the passive finger
J Comput Neurosci (2011) 30:45–67 63
(e.g. anatomical) knowlegde, may yield a method thatis more robust in the face of unkown and hard todetermine values for d and τ .
3.3 Effective connectivity at the MEG sensor level
Motor evoked f ields Self paced lifting of the right orleft index fingers in a self chosen sequence resultedin robust motor evoked fields, that were compatiblewith motor evoked fields reported in the literature(Mayville et al 2005; Weinberg et al. 1990; Nagamine etal. 1996; Pedersen et al. 1998) (Fig. 7). We observed aslow readiness field at sensors over contralateral motorcortices starting approximately 350 ms before onsetof EMG activity and a pronounced reversal of fieldpolarity during movement execution (data not shown).
Movement related ef fective connectivity As expected,effective connectivity from sensors over contralateralmotor cortices was significantly larger to EMG elec-trodes over the muscle of the moved finger than tothe EMG electrode over the muscle of the non-movedfinger (Fig. 8). Unexpectedly however, effective con-nectivity from ipsilateral motor cortices was also sig-nificantly larger to the EMG electrodes over the muscleof the moved finger than to the EMG electrode over themuscle of the non-moved finger. Effective connectivitywas never larger from any sensor over motor cortices tothe EMG electrodes over the muscle of the non-movedfinger.
4 Discussion
Transfer entropy as a tool to quantify ef fective connec-tivity In the present study we aimed to demonstratethat TE is a useful addition to existing methods forthe quantification of effective connectivity. We arguedthat existing methods like GC, that are based on linearstochastic models of the data, may have difficultiesdetecting purely non-linear interactions, such asinverted-U relationships. Here, we could show thattransfer entropy reliably detected effective connectivitycorrectly when two signals were coupled by a quadratic,i.e. purely non-linear, function (Fig. 2). Particularly rel-evant for neural interactions, we have also shown thatcouplings mediated by threshold or sigmoidal functionsare correctly captured by TE.
Furthermore, we extended the original definition ofTE to deal with long interaction delays and demon-strated that TE detected effective connectivity correctlywhen the coupling of two signals was mediated by
multiple interactions that spanned a range of latencies(Fig. 3).
Moreover, we considered the problem of volumeconduction and showed that TE robustly detectedeffective connectivity when only linear mixtures of theoriginal coupled signals were available (Figs. 4 and 5),if signals were not too close to being identical. Inaddition, if the two measurements reflected a com-mon underlying source signal (’common drive’) buthad different levels of measurement noise added, TEin combination with a test on time shifted data, cor-rectly rejected the hypothesis of effective connectivitybetween the two measurement signals, in contrast to anaive application of GC (Nolte et al. 2008). Therefore,TE in combination with this test is well applicableto EEG and MEG sensor-level signals, where linearinstantaneous mixing is inherent in the measurementmethod. However, without the additional test on timeshifted data, TE had a non-negligible rate of falsepositives detections of effective connectivity. The originof these false positives can be understood as follows.Theoretically transfer entropy is zero in the absence ofcausality, i.e. when processes are fully independent—asshould be the case for surrogate data. TE is also zerofor identical copies of a single signal, as required froma causality measure, when driver and response systemcannot be distinguished. Here, we considered the caseof volume conduction of a single signal onto two sen-sors in the presence of additional noise. Hence, the useof surrogate data for a test of the causality hypothesisinevitably leads to the comparison of two (noisy) zerosand false positives. Because of this difficulty we sug-gest to perform the time-shift test whenever multipleobservations of a single source signal are likely to bepresent in the data, as is the case for EEG and MEGmeasurements.
Last but not least, we proposed TE as a tool for theexploratory investigation of effective connectivity, be-cause it is a model-free measure based on informationtheory. Complicated types of coupling such as cross-frequency phase coupling (Palva et al. 2005) shouldbe readily detectable without prior specification, e.g.the coupling via a quadratic function—as investigatedhere—, introduces a frequency doubled (and distorted)input to the target signal. Nevertheless it was read-ily detected by TE. While the argument on model-freeness holds theoretically, any practical implemen-tation comes with certain parameters that have to beadapted to the data empirically, such as the correctchoice of a delay τ and the number of dimensions dused for delay embedding. In addition, the implemen-tation of TE proposed here incorporates a parameterfor the prediction time u to adapt the analysis for cases
64 J Comput Neurosci (2011) 30:45–67
where a long interaction delay is present. If chosen adhoc these parameters amount to a sort of model forthe data. To keep the method model-free we thereforeproposed to scan a sufficiently large parameter spaceon pilot data before analyzing the data of interest or toscan the parameter space and to correct for the arisingmultiple comparison problem later on, during statisticaltesting.
To handle the estimation of TE, the parameter scan-ning and the statistical testing, including the shift-test,we implemented the proposed procedure in the formof a convenient open-source MATLAB toolbox for theFieldtrip data format that is available from the authors(Lindner et al. 2009).
Limitations Despite the above-mentioned merits, theTE method also has limitations that have to be consid-ered carefully to avoid misinterpretations of the results:
We note that model-freeness is not always an ad-vantage. In contrast to model-based methods, the de-tection of effective connectivity via TE does not entailinformation on the type of interaction. This fact has twoimportant consequences. First, the absence of a specificmodel of the interaction leads to a high sensitivity forall types of depedencies between two time-series. Thisway, trivial (nuisance) dependencies, might be detectedby testing against surrogates. This is bound to happen ifthese dependencies are not kept intact when creatingthe surrogate data. Second, the specific type of inter-action must be separately assessed post hoc by usingmodel based methods, after the presence of effectiveconnectivity was established using transfer entropy. Inprinciple the analysis of effective connectivity using TE,and the post-hoc comparison of signal pairs with andwithout significant interaction in an exploratory searchof the actual mechanism of this interaction are possiblein the same dataset. This is because these two questionsare orthogonal. However, the relationship between sig-inificant effective connectivity—detected by TE—and aspecific mechanism of the interaction needs to be testedon independent data.
Another limitation is that false positive reports arepossible when the embedding parameters for the re-construction of the state space are not chosen correctly.We therefore suggest to use TE with a careful choiceof parameters, especially with respect to τ , and onlyafter checking that the data to be analyzed meets cer-tain characteristics. In the following we list a numberof characteristics to be considered. First, strong non-stationarities in the data can make impossible to av-erage over time to reliably estimate the probabilitydensities on which TE is based. Consequently, TEshould only be used on data of sufficient length that
show at most weak non-stationarities. For an approachto overcome this limitation problem by using the trialstructure of data sets see Gomez-Herrero et al. (2010).Second, in this work we have only assessed pairwiseinteractions. Although a fully multivariate extensionis conceptually possible (Gomez-Herrero et al. 2010;Lizier et al. 2008), practical data lengths and computingtime restrict its use. Third, TE analysis is difficult tointerpret when signals have a different physical originsuch as for example a chemical concentration and anelectric field. The reason is that even though the signalsentering the TE analysis are z-scored to obtain a certainnormalization, there is no clear physical meaning ofdistance in the joint space of the signals, and conse-quently, no a priori justification to use any particularcoarse-graining box in the two directions. Since theresults of TE are sensitive to the use of different coarse-graining scales in the two directions, the meaning ofany numerical estimate of TE for signals of differentphysical origin is difficult to establish. Finally, if theinteraction to be captured is known to be linear, thenthe use of linear approaches is fully justified and usuallyoutperforms TE in aspects such as computing time anddata-efficiency. Last but not least we should commenton some general limitations related to the concept ofcausality as defined by Wiener. It is important to notethat Wiener’s definition does not include any interven-tions to determine causality, i.e. it describes observa-tional causality. Methods based on Wiener’s principlesuch as GC, TE share certain limitations:
1. The decsription of all system involved has to becausally complete, i.e. there must not be unob-served common causes that do not enter the analy-sis.
2. If two systems are related by a deterministic map,no causality can be inferred. This would excludesystems exhibiting complete synchronization, forexample. Technically this is reflected in Eq. (4):For TE to be well defined the probability densi-ties and their logarithms must exist. Therefore δ-distributions in the joint embedding space of twosignals, which are equivalent to deterministic mapsbetween these signals, are excluded.
3. The concept of observational causality rests onthe axiom that the past and present may causethe future but the future may not cause the past.For this axiom to be useful observations must bemade at a rate that allows a meaningful distinctionbetween past, present and future with respect tothe interaction delays involved. This means thatinteractions that take place on a timescale faster
J Comput Neurosci (2011) 30:45–67 65
than the sampling rate must be missed in methodsbased on observational causality.
Application of TE to MEG recordings in a motor taskAs a proof-of-principle, we applied TE to MEG datarecorded during self paced finger lifting. The analysisof the effective connectivity from MEG to EMG signalswas performed without the recommended rectificationof the EMG signal (Myers et al. 2003) to proovethat TE could perform the analysis well without thisstep. Our expectations of stronger effective connec-tivity from contralateral motor cortex to the movedfinger were met for both fingers in both investigatedsubjects. Surprisingly, however, we also found strongereffective connectivity from ipsilateral motor cortex tothe moved finger. It is not clear at present whether thiseffective connectivity reflected an indirect interaction:Contralateral motor cortex may drive both, ipsilateralcortex and the muscles of the moved finger, albeit withstrongly differing delays. In this case, TE may erro-neously detect effective connectivity from ipsilateralcortex to the muscle, as discussed above. Additionalanalyses, quantifying the coupling between the twomotor cortices will be necessary to clarify this issue. Asdiscussed below, these analyses should preferentiallybe performed using a multivariate extension of the TEmethod.
Comparison to existing literature The application ofnon-linear methods to detect effective connectivity inneuroscience data has been suggested before: One ofthe earliest attempts to extend GC to the non-linearcase and to apply it to neurophysiological data waspresented by Freiwald et al. (1999). They used a lo-cally linear, non-linear autoregressive (LLNAR) modelwhere time varying autoregression coefficients wereused to capture non-linearities. This model was onlytested, however, on simulations of unidirectionally andlinearly coupled signals and correctly identified thecoupling as unidirectional and as linear. No attemptwas made to validate the model on simulations ofexplicitly non-linear directed interactions. Applicationto EEG data from a patient with complex partialseizures indicated non-linear coupling of the signalsmeasured at electrode positions C3 and C4. Anothertest on local field potential (LFP) data recorded in theanterior inferotemporal cortex (macaque area TE) ofthe macaque monkey however detected no indicationof a non-linear interaction. We add to these resultsby demonstrating that also purely non-linear (square,threshold) interactions are reliably detected using TEin combination with appropriate statistical testing andby demonstrating that interactions can also be found
in MEG and EMG data, even when omitting the usualrectification of the EMG. Chávez et al. (2003) used TEon data from an epileptic patient and also proposed astatistical test based on block-resampling of the datathat is similar to the trial shuffling approach used here.They found that TE with a fixed prediction time and afixed inclusion radius for neighbor search was able todetect the directed linear and non-linear interactionsfor the simulated models. Our findings are in agree-ment with these results. In addition, we demonstratedthat TE also detects directed non-linear interactionsfor biologically plausible data with 1/f characteristicsand a range of interaction delays instead of a singledelay. Hinrichs et al. (2008) used a measure that is verysimilar to transfer entropy as it was investigated here.However, in contrast to our study they substituted thetime-consuming estimation of probability densities bykernel-based methods with a linear method based onthe data covariance matrices. As explicitely stated inthe mathematical appendix of Hinrichs et al. (2008) thiseffectively limits the detection of directed interactionsto linear ones. Here, we demonstrate that, while beingrelatively time consuming, a kernel based estimation ofthe required probability densities is feasible using theKraskov-Stögbauer-Grassberger estimator (Kraskovet al. 2004), even for a dimensionality of five and higher.We note however, that the amount of data necessaryfor these estimations may not always be available andthat the achievable ‘temporal resolution’ is limited bythis factor. Interestingly, scanning of the predictiontime u, revealed an optimal interaction delay in theMEG/EMG data of around 16 ms, in accordance withtheir findings.
Outlook As demonstrated in this study TE is a usefultool to quantify effective connectivity in neurosciencedata. Its ability to detect purely non-linear interactionsand to operate without the specification of one ormore a priori models make it particularly useful forexploratory data analysis, but its use is not limited tothis application. The implementation of TE estimationused here only considered pairs of signals, i.e. it isa bivariate method. Direct and indirect interactionsmay, therefore, not be separated well. However, anextension to the multivariate case is possible as notedbefore (e.g. Chávez et al. 2003) and is currently underinvestigation. Its application to cellular automata byLizier and colleagues have already revealed interestinginsights into the pattern formation and informationflow in these models of complex systems (Lizier et al.2008).
The problem of direct versus indirect interactionscan also be ameliorated for the case of MEG and EEG
66 J Comput Neurosci (2011) 30:45–67
data by performing the analysis at the level of sourcetime-courses obtained from a suitable source analysismethod. Using source level time-courses will reduce thenumber of signals for analysis. A post hoc analysis ofthe obtained reduced network of effective connectivtyby DCM may be possible then. Using source leveltime-courses will also improve the interpretability ofthe obtained effective connectivities compared to thoseat the sensor level. This is because for a given causalinteraction observed at the sensor level any of themultiple sources reflected in the sensor signal may beresponsible for the observed effective connectivity.
Although the estimation of TE presented here isgeared at continuous data TE has found application inthe analysis of spiking data as reported in Gourvitchand Eggermont (2007). The particularities to estimateTE from point processes can be found there. Thus, bothmacroscopic (fMRI, EEG/MEG) and more local sig-nals (LFP, single unit activity) can be readily analizedin the common framework of TE. In the future, it willbe interesting to compare the effective connectivitiesfor a variety of temporal and spatial scales as revealedby TE.
Conclusion Transfer entropy robustly detectedeffective connectivity in simulated data both forcomplex internal signal dynamics (1/ f ) and for stronglynon-linear coupling. Detection of effective connectivitywas possible without specifying an a priori model. Withthe use of an additional test for linear instantaneousmixing it was robust against false positives due tosimulated volume conduction. Therefore it is notonly applicable for invasive electrophysiological databut also for EEG and MEG sensor-level analysis.Analysis of MEG and EMG sensor-level data recordedin a simple motor task data revealed the expectedconnectivity, even without rectification of the EMGsignal. We therefore propose TE as a useful tool forthe analysis of effective connectivity in neurosciencedata.
Acknowledgements The authors would like to thank ViolaPriesemann from the Max Planck Institute for Brain Research,Frankfurt, for valuable comments on this manuscript, GermanGomez Herrero from the Technical University of Tampere,Wei Wu from the Humboldt-Universität in Berlin, MikhailProkopenko from the CSIRO in Sydney, and Prof. JochenTriesch from the Frankfurt Institute for Advanced Studies(FIAS) for stimulating discussions, and Sarah Straub from theDepartment of Psychology, University of Regensburg for assis-tance in data acquisition.
Open Access This article is distributed under the terms of theCreative Commons Attribution Noncommercial License whichpermits any noncommercial use, distribution, and reproduction
in any medium, provided the original author(s) and source arecredited.
References
Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996). Dy-namics of ongoing activity: Explanation of the large variabil-ity in evoked cortical responses. Science, 273(5283), 1868–1871.
Cao, L. (1997). Practical method for determining the minimumembedding dimension of a scalar time series. Physica, A, 110,43–50.
Chávez, M., Martinerie, J., & Quyen, M. L. V. (2003). Statisticalassessment of non-linear causality: Application to epilepticeeg signals. Journal of Neuroscience Methods, 124(2), 113–128.
Cormen, T., Leiserson, C., Rivest, R., & Stein, C. (2001). Intro-duction to algorithms. MIT Press and McGraw-Hill.
Donoghue, J. P., Sanes, J. N., Hatsopoulos, N. G., & Gal, G.(1998). Neural discharge and local field potential oscillationsin primate motor cortex during voluntary movements. Jour-nal of Neurophysiology, 79(1), 159–173.
Erimaki, S., & Christakos, C. N. (2008). Coherent motor unitrhythms in the 6–10 hz range during time-varying volun-tary muscle contractions: Neural mechanism and relationto rhythmical motor control. Journal of Neurophysiology,99(2), 473–483. doi:10.1152/jn.00341.2007.
Freiwald, W. A., Valdes, P., Bosch, J., Biscay, R., Jimenez, J. C.,Rodriguez, L. M., et al. (1999). Testing non-linearity anddirectedness of interactions between neural groups in themacaque inferotemporal cortex. Journal of NeuroscienceMethods, 94(1), 105–119.
Friston, K. (1994). Functional and effective connectivity in neu-roimaging: A synthesis. Human Brain Mapping, 2, 56–78.
Friston, K. J., Harrison, L., & Penny, W. (2003). Dynamic causalmodelling. NeuroImage, 19(4), 1273–1302.
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002). Thresh-olding of statistical maps in functional neuroimaging us-ing the false discovery rate. NeuroImage, 15(4), 870–878.doi:10.1006/nimg.2001.1037.
Gomez-Herrero, G., Wu, W., Rutanen, K., Soriano, M. C., Pipa,G., & Vicente, R. (2010). Assessing coupling dynamics froman ensemble of time series. arXiv:1008.0539v1.
Gourvitch, B., & Eggermont, J. J. (2007). Evaluating informationtransfer between auditory cortical neurons. Journal of Neu-rophysiology, 97(3), 2533–2543. doi:10.1152/jn.01106.2006.
Granger, C. (1980). Long memory relationships and the aggrega-tion of dynamic models. Journal of Econometrics, 14, 227–238.
Granger, C. W. J. (1969). Investigating causal relations by econo-metric models and cross-spectral methods. Econometrica, 37,424–438.
Gross, J., Timmermann, L., Kujala, J., Dirks, M., Schmitz, F.,Salmelin, R., et al. (2002). The neural basis of intermittentmotor control in humans. Proceedings of the National Acad-emy of Sciences of the United States of America, 99(4), 2299–2302. doi:10.1073/pnas.032682099.
Hinrichs, H., Noesselt, T., & Heinze, H. J. (2008). Directed in-formation flow: A model free measure to analyze causalinteractions in event related eeg-meg-experiments. HumanBrain Mapping, 29(2), 193–206. doi:10.1002/hbm.20382.
Hlavackova-Schindler, K., Palus, M., Vejmelka, M., &Bhattacharya, J. (2007). Causality detection based oninformation-theoretic approaches in time series analysis.Physics Reports, 441, 1–46.
Kaiser, A., & Schreiber, T. (2002). Information transfer in con-tinuous processes. Physica, D, 110, 43–62.
Kantz, H., & Schreiber, T. (1997). Nonlinear time series analysis.Cambridge University Press.
Kozachenko, L., & Leonenko, N. (1987). Sample estimate ofentropy of a random vector. Problems of Information Trans-mission, 23, 95–100.
Kraskov, A., Stögbauer, H., & Grassberger, P. (2004). Estimatingmutual information. Physical Review. E, Statistical, Nonlin-ear, and Soft Matter Physics, 69(6 Pt 2), 066,138.
Lindner, M., Vicente, R., & Wibral, M. (2009). Trentool—the transfer entropy toolbox. http://www.michael-wibral.de/TRENTOOL. Accessed 7 August 2010.
Lizier, J., Prokopenko, M., & Zomaya, A. (2008). Local informa-tion transfer as a spatiotemporal filter for complex systems.Physical Review. E, 77, 026,110.
Mayville, J. M., Fuchs, A., & Kelso, J. A. S. (2005). Neuromag-netic motor fields accompanying self-paced rhythmic fingermovement at different rates. Experimental Brain Research,166(2), 190–199. doi:10.1007/s00221-005-2354-2.
Merkwirth, C., Parlitz, U., Wedekind, I., Engster, D., &Lauterborn, W. (2009). Opentstool version 1.2 (2/2009).http://www.physik3.gwdg.de/tstool/index.html. Accessed 7August 2010.
Mima, T., Matsuoka, T., & Hallett, M. (2000). Functional cou-pling of human right and left cortical motor areas demon-strated with partial coherence analysis. Neuroscience Letters,287(2), 93–96.
Murthy, V. N., & Fetz, E. E. (1996). Oscillatory activity in senso-rimotor cortex of awake monkeys: Synchronization of localfield potentials and relation to behavior. Journal of Neuro-physiology, 76(6), 3949–3967.
Myers, L. J., Lowery, M., O’Malley, M., Vaughan, C. L.,Heneghan, C., Gibson, A. S. C., et al. (2003). Rectificationand non-linear pre-processing of emg signals for cortico-muscular analysis. Journal of Neuroscience Methods, 124(2),157–165.
Nagamine, T., Kajola, M., Salmelin, R., Shibasaki, H., & Hari, R.Movement-related slow cortical magnetic fields and changesof spontaneous meg- and eeg-brain rhythms. Electroen-cephalography and Clinical Neurophysiology, 99(3), 274–286.
Nalatore, H., Ding, M., & Rangarajan, G. (2007). Mitigating theeffects of measurement noise on Granger causality. Physi-cal Review. E, Statistical, Nonlinear and Soft Matter Physics,75(3 Pt 1), 031,123.
Nolte, G., Ziehe, A., Nikulin, V. V., Schloegl, A., Kraemer, N.,Brismar, T., et al. (2008). Robustly estimating the flow di-rection of information in complex physical systems. PhysicalReview Letters, 100(23), 234,101.
Paluš, M. (2001). Synchronization as adjustment of informationrates: Detection from bivariate time series. Physical Review.E, 63, 046,211.
Palva, J. M., Palva, S., & Kaila, K. (2005). Phase syn-chrony among neuronal oscillations in the human cortex.Journal of Neuroscience, 25(15), 3962–3972. doi:10.1523/JNEUROSCI.4250-04.2005.
Pedersen, J. R., Johannsen, P., Bak, C. K., Kofoed, B., Saermark,K., & Gjedde, A. (1998). Origin of human motor readi-ness field linked to left middle frontal gyrus by megand pet. NeuroImage, 8(2), 214–220. doi:10.1006/nimg.1998.0362.
Pereda, E., Quiroga, R., & Bhattacharya, J. (2005). Nonlinearmultivariate analysis of neurophysiological signals. Progressin Neurobiology, 77, 1–37.
Pogosyan, A., Gaynor, L. D., Eusebio, A., & Brown, P. (2009).Boosting cortical activity at beta-band frequencies slowsmovement in humans. Current Biology, 19(19), 1637–1641.doi:10.1016/j.cub.2009.07.074.
Ragwitz, M., & Kantz, H. (2002). Markov models from data bysimple nonlinear time series predictors in delay embeddingspaces. Physical Review. E, 65, 056201.
Reza, F. (1994). An introduction to information theory.Dover.
Schoffelen, J. M., Oostenveld, R., & Fries, P. (2008). Imag-ing the human motor system’s beta-band synchronizationduring isometric contraction. NeuroImage, 41(2), 437–447.doi:10.1016/j.neuroimage.2008.01.045.
Schreiber, T. (2000). Measuring information transfer. PhysicalReview Letters, 85(2), 461–464.
Shannon, C. (1948). A mathematical theory of communication.Bell System Technical Journal, 27, 379–423.
Swadlow, H. (1985). Physiological properties of individual cere-bral axons studied in vivo for as long as one year. Journal ofNeurophysiology, 54, 1346–1362.
Swadlow, H. (1994). Efferent neurons and suspected interneu-rons in motor cortex of the awake rabbit: Axonal properties,sensory receptive fields, and subthreshold synaptic inputs.Journal of Neurophysiology, 71, 437–453.
Swadlow, H., Rosene, D., & Waxman, S. (1978). Characteristicsof interhemispheric impulse conduction between the pre-lunate gyri of the rhesus monkey. Experimental Brain Re-search, 33, 455–467.
Swadlow, H., & Waxman, S. (1975). Observations on impulseconduction along central axons. Proceedings of the NationalAcademy of Sciences, 72, 5156–5159.
Takens, F. (1981). Dynamical Systems and Turbulence, Warwick1980. In Lecture Notes in Mathematics (Vol. 898, chap.).Detecting Strange Attractors in Turbulence (pp. 366–381).Springer.
Tognoli, E., & Scott Kelso, J. (2009). Brain coordination dynam-ics: True and false faces of phase synchrony and metastabil-ity. Progress in Neurobiology, 12, 31–40.
Victor, J. (2002). Binless strategies for estimation of informationfrom neural data. Physical review. E, 66, 051903.
Weinberg, H., Cheyne, D., Crisp, D. (1990). Electroencephalo-graphic and magnetoencephalographic studies of motorfunction. Advances in Neurology, 54, 193–205.
Wiener, N. (1956). The theory of prediction. In: E. F. Beckenbach(Ed.), In modern mathematics for the engineer. McGraw-Hill,New York.
Yerkes, R. M., & Dodson, J. D. (1908). The relation of strengthof stimulus to rapidity of habit-formation. Journal of Com-parative Neurology and Psychology, 18, 459.
