tspace.library.utoronto.ca...Dedication A Katam et Zia, qui font toute la diﬀ´erence, a...

Towards Robust Quantification of Cosmological Errors

by

Joachim Harnois-Deraps

A thesis submitted in conformity with the requirementsfor the degree of Doctor of Philosophy

Graduate Department of PhysicsUniversity of Toronto

Copyright c© 2013 by Joachim Harnois-Deraps

Abstract

Towards Robust Quantification of Cosmological Errors

Joachim Harnois-Deraps

Doctor of Philosophy

Graduate Department of Physics

University of Toronto

2013

The method of baryon acoustic oscillation (BAO) is among the best probes of the dark energy

equation of state, and worldwide efforts are being invested in order to perform measurements that are

accurate at the percent level. In current data analyses, however, estimates of the error about the BAO

are based on the assumption that the density field can be treated as Gaussian, an assumption that

becomes less accurate as smaller scales are included in the measurement. It was recently shown from

large samples of N-body simulations that the error bars about the BAO obtained this way are in fact

up to 15-20 per cent too small. This important bias has shaken the confidence in the way error bars are

calculated, and is motivating developments of analyses pipelines that include non-Gaussian features in

the matter density fields.

In this thesis, we propose general strategies to incorporate non-Gaussian effects in the context of a

survey. After describing the high performance N-body code that we used, we present novel properties of

the non-Gaussian uncertainty about the matter power spectrum, and explain how these combine with

a general survey selection function. Assuming that the non-Gaussian features that are observed in the

simulations correspond to those of Nature, this approach is the first unbiased measurement of the error

bar about the power spectrum, which simultaneously removes the undesired bias on the BAO error. We

then relax this assumption about the similitude of the non-Gaussian natures in simulations and data,

and develop tools that aim at measuring the non-Gaussian error bars exclusively from the data.

It is possible to improve the constraining power of non-Gaussian analyses with ‘Gaussianizations’

techniques, which map the observed fields into something more Gaussian. We show that two of such

techniques maximally recover degrees of freedom that were lost in the gravitational collapse. Finally,

from a large sample of high resolution N-body realizations, we construct a series of weak gravitational

lensing distortion maps and provide high resolution halo catalogues that are used by the CFTHLenS

community to calibrate their estimators and study many secondary effects with unprecedented accuracy.

ii

Dedication

A Katam et Zia, qui font toute la difference,

a Kerry-Anne, pour sa clarete et ses kilometres de temps,

a mon pere, qui est encore un papa,

a Luce et Alain, pour leur vitalite contagieuse,

a Catherine, pour ses encouragements qui dispersent la brume,

et, finalement, a Pierrette, dont le sourire nous suit partout.

Acknowledgements

I am deeply grateful to Dr. Ue-Li Pen, my research supervisor, whose far reaching vision and problem

solving skills still amaze me. In darker times, his confidence in the importance of my work put me back

on track; in brighter times, his enthusiasm got me involved in as many exciting projects I could handle.

I am also grateful to Dr. Ludovic Van Waerbeke and Sanaz Vafaei, who invited me three times at UBC

during the course of my degree; these travels brought colour and light in my graduate experience. I

would also like to thank Ting-Ting Lu, Olivier Dore, Hao-Ran Yu and Kiyoshi Masui, which helped me

in so many ways I can’t count. I would like to acknowledge the dedicated support from the technical staff

at CITA and SciNet, who were incredibly patient and caring whenever I needed help with computing

issues. Finally, acknowledgements that are specific to research projects and funding agencies are included

within the chapters of this thesis.

iii

Contents

1 Introduction 1

1.1 General Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Error Estimates in BAO Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 This Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 High Performance P3M N-body code: CUBEP3M 7

2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 Review of the Code Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 Poisson Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.5 Scaling Performances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.6 Accuracy and Systematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.6.1 Density and power spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.6.2 Mesh force at grid distances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6.3 Constraining redshift jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.7 Runtime Halo Finder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.8 Beyond the standard configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.8.1 Initial conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.8.2 Particle identification tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.8.3 Extended range of the pp force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3 Non-Gaussian Error Bars in Galaxy Surveys-I 45

3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

iv

3.3 Matter Power Spectrum from Galaxy Surveys . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.3.1 The optimal estimator of the power spectrum . . . . . . . . . . . . . . . . . . . . . 50

3.3.2 The FKP covariance matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.3.3 Non-Gaussian error bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.4 Measuring the Angular Dependence: the Method . . . . . . . . . . . . . . . . . . . . . . . 54

3.4.1 Cross-correlations from Fourier transforms . . . . . . . . . . . . . . . . . . . . . . . 54

3.4.2 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.4.3 Zero-lag point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.5 Validation of the Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.5.1 Testing C(k, k′, θ) with a Legendre polynomial . . . . . . . . . . . . . . . . . . . . 58

3.5.2 Testing C(k, k′, θ) with Poisson-sampled random fields . . . . . . . . . . . . . . . . 59

3.5.3 Testing C(k, k′, θ) with Gaussian random fields . . . . . . . . . . . . . . . . . . . . 60

3.6 Measuring the Angular Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.6.1 N-body simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.6.3 From C(ki, kj , θ) to C(ki, kj) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.7 Multipole Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.7.1 From C(ki, kj , θ) to Cℓ(ki, kj) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.7.2 Testing Cℓ with a Legendre polynomial, with Poisson and Gaussian distributions . 68

3.7.3 Measuring Cℓ(ki, kj) from simulations . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.7.4 Cℓ(k, k′) matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.8 Factorization of the Cℓ Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.8.1 Non-Gaussian Poisson noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.8.2 Recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.9 Measuring the Impact with Selection Functions . . . . . . . . . . . . . . . . . . . . . . . . 80

3.9.1 Factorization of the 6-dimensional covariance matrix . . . . . . . . . . . . . . . . . 81

3.9.2 The 2dFGRS selection function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.9.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.10 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3.11 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.A Legendre-Gauss Weighted Summation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.B Eigenvector of the Poisson noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

v

4 Non-Gaussian Error Bars in Galaxy Surveys-II 93

4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.3 Extracting and filtering C(k,k’): bootstrap approach . . . . . . . . . . . . . . . . . . . . . 97

4.3.1 Diagonal elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.3.2 Off-diagonal elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.4 Optimal Estimation of C(k, k′): Noise Weighting . . . . . . . . . . . . . . . . . . . . . . . 106

4.4.1 Noise-weighting NG(k,k’) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4.4.2 Wiener filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.4.3 Noise weighted Eigenvectors decomposition . . . . . . . . . . . . . . . . . . . . . . 109

4.5 Impact on Fisher Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4.6 In the Presence of Surveys Selection Functions . . . . . . . . . . . . . . . . . . . . . . . . 114

4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

5 Optimizing the Recovery of Fisher Information 120

5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.3 Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5.3.1 Numerical Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5.3.2 Density reconstruction algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

5.3.3 Wavelet non-linear Wiener filter (WNLWF) . . . . . . . . . . . . . . . . . . . . . . 124

5.3.4 Information recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.4.1 Density fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.4.2 Covariance matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

5.4.3 Fisher information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

5.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6 Gravitational Lensing Simulations 139

6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

6.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

6.3 Theory of Weak Lensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

6.4 Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

6.4.1 Constructing the lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

vi

6.4.2 N-Body simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

6.5 Testing the simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.5.1 Dark matter density power spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.5.2 Weak lensing power spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

6.5.3 Halo Catalogues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

6.6 Weak Lensing with 2-Point Correlation Functions . . . . . . . . . . . . . . . . . . . . . . . 159

6.6.1 Shear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

6.6.2 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6.7 Weak Lensing with Window-Integrated Correlation Functions . . . . . . . . . . . . . . . 165

6.8 Weak Lensing with Windowed Statistics on Convergence Maps . . . . . . . . . . . . . . . 167

6.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

7 Conclusion and Future Work 175

Bibliography 179

vii

Chapter 1

Introduction

1.1 General Context

The field of cosmology recently took an important turn with the realization that the global expansion

of the Universe seems to be accelerating. This groundbreaking discovery was first based on the redshift-

distance relation measured from a large sample of Type 1A supernovae, soon to be confirmed by other

independent techniques. The implications are profound and have major consequences on our under-

standing of the Universe: either the theory of General Relativity is wrong or incomplete, in which case

the observed acceleration is only apparent, or there exists a new substance, the dark energy, that drives

the acceleration. The second scenario is now considered the standard model of cosmology, in which

about 73 per cent of the density of the Universe is in the form of dark energy; 23 per cent is attributed

to dark matter, and the ordinary matter – galaxies, stars, gas, dust, etc. – occupies only four per cent.

The international community of physicists, cosmologists and astronomers soon realized that the most

efficient way to quantify and, hopefully, understand the nature of dark energy is to organize indepen-

dent measurements in coherent and complimentary ways. Based on current technology and theoretical

understanding, four techniques have been identified as the most apt to measure the amount and time

dependence of dark energy.These techniques are organized into series of measurements with increasing

precision and accuracy. We refer the reader to the Report on the Dark Energy Task Force Albrecht

et al. (2006) for a detailed discussion of this global plan. These are 1- detection of Type 1A supernovae,

2- measurement of the stretching of the baryonic acoustic oscillation (hereafter BAO), 3- measurement

of the growth rate of the large scale structures from galaxy clusters and 4- weak gravitational lensing

analyses. Many first and second generations of these measurements are now completed, including recent

1

Chapter 1. Introduction 2

results from the Hubble Space Telescope Cluster Supernova Survey1, from the Canada-France-Hawaii

Telescope Lensing Survey2, from the SuperNova Legacy Survey3, from the Sloan Digital Sky Survey-III

Supernova Survey 4 and BOSS5, from the WiggleZ Survey6 or from the 6 degree Field Galaxy Survey7.

New experiments such as the Square Kilometre Array8, Euclid9, the Dark Energy Survey10 or the Kilo

Degree Survey11 are underway and new results are expected to be coming out over the next decades.

So far, the combined analyses suggest that the dark energy component is constant both in time and in

space, at least within the estimated error bars.

While the uncertainty budget in most current analyses is dominated by statistics and systematics,

upcoming surveys will reverse this situation. As a matter of fact, these are designed such that the

instrumental noise and the statistical uncertainty in the data will drop significantly, sometimes below

the current theoretical uncertainty. This situation is forcing the theorists to improve their models such

as to match the forecasted level of accuracy; all of the approximations currently made in the analyses

pipelines need to be revisited. This is true both for those concerning the measurement of the mean

itself and for the estimate of the associated uncertainty: without robust error bars, the data could be

consistent with a large number of theoretical predictions. In such a situation, the constraining power of

the measurement about parameters in the underlying model is weak.

1.2 Error Estimates in BAO Analyses

The technique of BAO aims at measuring, at different redshift, a small oscillating signal in the matter

power spectrum. The stretching of these wiggles is directly linked to the expansion of the Universe,

from which we infer the amount of dark energy. Similarly, the error on the amplitude of the matter

power spectrum propagates onto the error on the BAO measurement, then on dark energy. In that error

calculation, however, most data analyses rely on simplifying assumptions about the statistical nature

of the underlying matter density field. In particular, the matter distributions are typically regarded as

observing Gaussian statistics, in which all measurements in Fourier space are uncorrelated. In fact, a

large fraction of the density field has undergone non-linear gravitational collapse, which is known to

1supernova.lbl.gov2www.cfhtlens.org3www.cfht.hawaii.edu/SNLS4www.sdss.org/supernova/aboutsupernova.html5www.sdss3.org/surveys/boss.php6wigglez.swin.edu.au7www.aao.gov.au/6dFGS8www.skatelescope.org/9sci.esa.int/science-e/www/object/index.cfm?fobjectid=46680

10www.darkenergysurvey.org/11kids.strw.leidenuniv.nl/


produce non-Gaussian features: the Fourier modes couple together, an effect which both increases and

correlates the error bars.

Rimes & Hamilton (2005) have shown from a large sample of N-body simulations that neglecting

this interplay results in a wrong estimate of the available information about the matter power spec-

trum: a Gaussian treatment overshoots the simulation by an order of magnitude at the smallest scales

relevant for BAO. The reason for this is that the coupling of Fourier modes decreases the effective num-

ber of degrees of freedom in a measurement. When propagated to an actual BAO measurement, the

Gaussian/non-Gaussian discrepancy is milder, but still significant: Ngan et al. (2012) measured from

N-body simulations that neglecting non-Gaussian effects yields error bars on the BAO that are too low

by 15-20 per cent. In this context, it seems surprising that data analyses are still treating the fields as

Gaussian. Put together, these realizations are thereby affecting our confidence on the quoted uncertainty

about the dark energy. There is currently an important gap between the way we treat the data and the

simulations, and so far very little effort has been invested to connect these two realms.

In non-Gaussian treatments of N-body simulations, the strategy is straightforward: one needs to

model accurately the full covariance matrix about the amplitude of the matter power spectrum, then to

propagate this uncertainty onto a BAO measurement, typically with a Fisher matrix. Performing the

same operation in the data is more challenging for many reasons. First, observations are subject to a

survey selection function that needs to be convolved with the power spectrum and with its uncertainty.

The power spectrum is isotropic, hence the convolution is done only in the radial direction. In the case of

the uncertainty, neither the selection function nor the covariance matrix are isotropic. The convolution

is much harder to calculate, and involves angular integrations as well, which had never been attempted.

In particular, one therefore needs to understand the angular dependence of the error, i.e. how does it

behaves as the angle between the two Fourier modes changes. This is, so far, the only way one can

estimate the non-Gaussian error bars on the BAO scale from the data that is free of the 15-20 per cent

bias.

Filling this gap between simulations and data analyses is what the core of this thesis is about. After

a detailed description of one of the most performant public N-body code (Chapter 2) we develop a

technique to estimate unbiased error bars on the matter power spectrum, in the presence of a survey

selection function, in order to provide better estimate of the uncertainty about the BAO scale and on

dark energy (Chapter 3). In that process, we measure the angular dependence of the covariance matrix

about the matter power spectrum, and parametrize the deviations observed in our N-body simulation

from Gaussian statistics. Since there are many ways for a field to be non-Gaussian, it is possible that

the non-Gaussian features observed in the simulation do not correspond to those of the actual Universe.


For this purpose, we set the first stone towards internal estimation of non-Gaussian error bars in data

analyses, i.e. without relying on simulations at all (Chapter 4). To do so, we show that the full covariance

matrix can be estimated from a handful of realizations, with the help of a combination of noise filtering

techniques. Although most of the non-Gaussian effects are observed in scales barely resolved in current

surveys, these will become available in the next generation of experiments, in which the statistical and

instrumental errors are expected to drop massively. It is thus crucial to improve our methods such as to

get rid of the 15-20 per cent bias that currently exists about the error on the BAO scale.

By recognizing the non-Gaussian nature of the density field, it has been shown that it is possible to

enhance the constraining power of BAO analyses by applying different operations on the field. These

techniques are traditionally referred to ‘Gaussianization’ strategies, as they map the observed density

field into something more Gaussian, thereby increasing the information about the power spectrum.

Whereas these were always studied separately, we show that it is possible to combine some of them and

optimize the recovery of information (Chapter 5).

1.3 This Thesis

The five following chapters constitute the core of this document; each of these is a re-edition of a

published – or at least public – article, and as such is self-contained and written at the peer-reviewed

journal level. All the necessary theoretical background is introduced therein, and each chapter ends with

discussions of the results, concluding remarks and appendices when necessary. It is assumed throughout

this thesis that the reader is familiar with basic error analysis techniques and first order perturbation

theory in the cosmological context; we refer the reader to any recent cosmology textbook at the graduate

student level for more details on the theoretical background. In the final part of this introduction, I

provide a brief summary of each chapter, and contrast in some occasion my own contribution from that

of co-authors.

Chapter 2 (Harnois-Deraps et al., 2012a) describes in details CUBEP3M, a high performance N-body

simulator that is massively utilized throughout the rest of the thesis. Although I did not write this

FORTRAN90 code all by myself, I have brought significant contributions in development, extensions,

debugging and testing of both the code itself and many of the pre- and post-processing utilities. This

highly parallel cosmological code reads in a set of initial conditions and solves Poisson’s equation on a

two-level mesh, with an exact pairwise interaction for particles that are nearby. One of the main project I

had was to extend this pairwise force to an arbitrary distance, which allows for greater accuracy. Among

other projects, I have also incorporated a system of unique particle identification tags that allows to


track down trajectories of individual objects across multiple time steps. I have also brought back an old

deprecated interface with a MHD-TVD module, I have found and removed bugs in the post-processing

power spectrum code, created subroutines to test the force against Newton’s predictions in more efficient

way, and perform a large series of systematic tests.

Chapter 3 (Harnois-Deraps & Pen, 2012) and Chapter 4 present calculations that aim at removing

the bias in the estimate of the uncertainty on the dark matter power spectrum, that is responsible for

the 15-20 per cent under shoot in the error about the BAO measurement. We provide a general recipe

that estimates correctly the full non-Gaussian covariance matrix about the matter power spectrum,

including the required information about the angular dependence of the error, and in presence of a

survey selection function (Chapter 3). We then progress towards internal analyses and present strategies

to perform similar measurements exclusively from the data (Chapter 4).

Chapter 5 (Harnois-Deraps et al., 2012c) examines how we can reduce the uncertainty about the

amplitude of the power spectrum, once its non-Gaussian error is correctly measured. It makes use of

two complementary techniques that attempt to ‘undo’ the correlation. The first is a density reconstruc-

tion algorithm that propagates the particles (or haloes) outside the gravitational potential using linear

perturbation theory. This effectively maps the matter field to an earlier state, where it was actually less

correlated. The second technique applies a non-linear Wiener filter on the density field and factorizes

out the part that is highly-correlated. The filter, constructed in wavelet space, was mostly developed by

a collaborator, Hao-Ran Yu, which visited the Canadian Institute for Theoretical Astrophysics for a full

year. We show that we are able to recover up to five times more degrees of freedom in a power spectrum

measurement, compared to the straightforward non-Gaussian analysis. When applied to actual data,

these techniques should also combine well and provide the tightest constraints about BAO and thus dark

energy.

In Chapter 6 (Harnois-Deraps et al., 2012b), we navigate away from the realm of BAO and describe

the construction of mock galaxy catalogues that served to calibrate weak lensing analysis pipelines

from the final data release of the CFHTLenS analyses (MNRAS submitted). Also part of that chapter

is an important measurement of the correlated uncertainty about a series of detection signals, both in

configuration and Fourier space. Finally, we constructed a halo catalogue accurate at the sub-arc minute

level that will be used in the near future to test secondary effects in weak lensing analyses. Most of the

weak lensing estimators beyond the convergence and shear distortion were constructed by Dr. Sanaz

Vafaei and Dr. Ludovic van Waerbeke from the University of British Columbia.

Since detailed results can be found in the final section of each chapter, this thesis concludes with a

very brief summary of the results in Chapter 7, and a discussion about some of their implications. I


finally brush a quick portrait of open issues, projects in progress and personal intentions for the next

few years.

Chapter 2

High Performance P3M N-body

code: CUBEP3M

2.1 Summary

This paper presents CUBEP3M, a high performance, publicly-available, cosmological N-body code and

describes many utilities and extensions that have been added to the standard package, including a

runtime halo finder, a non-Gaussian initial conditions generator, a tuneable accuracy, and a system of

unique particle identification. CUBEP3M is fast, has a memory imprint up to three times lower than

other widely used N-body codes, and has been run on up to 20, 000 cores, achieving close to ideal weak

scaling even at this problem size. It is well suited and has already been used for a broad number of

science applications that require either large samples of non-linear realizations or very large dark matter

N-body simulations, including cosmological reionization, baryonic acoustic oscillations, weak lensing or

non-Gaussian statistics. We discuss the structure, the accuracy, any known systematic effects, and the

scaling performance of the code and its utilities, when applicable.

2.2 Introduction

Many physical and astrophysical systems are subject to non-linear dynamics and rely on N-body sim-

ulations to describe the evolution of bodies. One of the main fields of application is the modelling of

large scale structures, which are driven by the sole force of gravity. Recent observations of the cosmic

microwave background (Komatsu et al., 2009, 2011), of galaxy clustering (York et al., 2000; Colless

7

Chapter 2. High Performance P3M N-body code: CUBEP3M 8

et al., 2003; Schlegel et al., 2009b; Drinkwater et al., 2010) of weak gravitational lensing (Heymans &

CFHTLenS Collaboration, 2012; Sheldon et al., 2009) and of supernovae redshift-distance relations all

point towards a standard model of cosmology, in which dark energy and collision-less dark matter occupy

more than 95 per cent of the total energy density of the universe. In such a paradigm, pure N-body

codes are perfectly suited to describe the dynamics, as long as baryonic physics are not very important,

or at least we understand how the baryonic fluid feeds back on the dark matter structure. The next

generation of measurements aims at constraining the cosmological parameters at the per cent level, and

the theoretical understanding of the non-linear dynamics that govern structure formation heavily relies

on numerical simulations.

For instance, a measurement of the baryonic acoustic oscillation (BAO) dilation scale can provide

tight constraints on the dark energy equation of state (Eisenstein et al., 2005; Tegmark et al., 2006;

Percival et al., 2007; Schlegel et al., 2009b). The optimal estimates of the uncertainty require the

knowledge of the matter power spectrum covariance matrix, which is only accurate when measured from

a large sample of N-body simulations (Rimes & Hamilton, 2005; Takahashi et al., 2009, 2011). For

the same reasons, the most accurate estimates of weak gravitational lensing signals are obtained by

propagating photons in past light cones that are extracted from simulated density fields (Vale & White,

2003; Sato et al., 2009; Hilbert et al., 2009). Another area where large-scale N-body simulations have

been instrumental in recent years are in simulations of early cosmic structures and reionization (e.g. Iliev

et al., 2006; Zahn et al., 2007; Trac & Cen, 2007; Iliev et al., 2012). The reionization process is primarily

driven by low-mass galaxies, which for both observational and theoretical reasons, need to be resolved

in fairly large volumes, which demands simulations with a large dynamical range.

The basic problem that is addressed with N-body codes is the time evolution of an ensemble of

N particles that is subject to gravitational attraction. The brute force calculation requires O(N2)

operations, a cost that exceeds the memory and speed of current machines for large problems. Solving

the problem on a mesh (Hockney & Eastwood, 1981) reduces to O(N logN) the number of operations,

as it is possible to solve for the particle-mesh (PM) interaction with fast Fourier transforms techniques

that rely on high performance libraries such as FFTW (Frigo & Johnson, 2005).

With the advent of large computing facilities, parallel computations have now become common

practice, and N-body codes have evolved both in performance and complexity. Many have opted for

‘tree’ algorithms, including GADGET (Springel et al., 2001; Springel, 2005), PMT (Xu, 1995), GOTPM

(Dubinski et al., 2004), or adaptive P3M codes like Hydra (Couchman et al., 1995), in which the local

resolution increases with the density of the matter field. These often have the advantage to balance

the work load across the computing units, which enable fast calculations even in high density regions.


The drawback is a significant loss in speed, which can be only partly recovered by turning off the tree

algorithm. The same reasoning applies to mesh-refined codes (Couchman, 1991), which in the end are

not designed to perform fast PM calculations on large scales.

Although such codes are needed to study systems that are spatially heterogeneous like individual

clusters or haloes, AGNs or other compact objects, many applications are interested in studying large

cosmological volumes in which the matter distribution is rather homogeneous. In such environments,

the load balancing algorithm can be removed, and the effort can thereby be put towards speed, memory

compactness and scalability. PMFAST (Merz et al. (2005), MPT hereafter) was one of the first code

designed specifically to optimize the PM algorithm, both in terms of speed and memory usage, and

uses a two-level mesh algorithm based on the gravity solver of Trac & Pen (2003). The long range

gravitational force is computed on a grid 43 times coarser, such as to minimize the MPI communication

time and to fit in system’s memory. The short range is computed locally on a finer mesh, and only

the local sub-volume needs to be stored at a given time, allowing for OPENMP parallel computation.

This setup enables the code to evolve large cosmological systems both rapidly and accurately, even on

relatively modest clusters. One of the main advantages of PMFAST over other PM codes is that the

number of large arrays is minimized, and the global MPI communications are cut down to the minimum:

for passing particles at the beginning of each time step, and for computing the long range FFTs.

Since its first published version, PMFAST has evolved in many aspects. The first major improvement

was to transform the volume decomposition in multi-node configurations from slabs to cubes. The

problem with slabs is that they do not scale well to large runs: as the number of cells per dimension

increases, the thickness of each slab shrinks rapidly, until it reaches the hard limit of a single cell layer.

With this enhancement, the code name was changed to CUBEPM. Soon after, it incorporated particle-

particle (pp) interactions at the sub-grid level, and was finally renamed CUBEP3M. The public package

now includes a significant number of new features: the pp force can be extended to an arbitrary range,

the size of the redshift jumps can be constrained for improved accuracy during the first time steps, a

runtime halo finder has been implemented, the expansion has also been generalized to include a redshift

dependent equation of state for dark energy, there is a system of unique particle identification that

can be switched on or off, and the initial condition generator has been generalized as to include non-

Gaussian features. PMFAST was equipped with a multi-time stepping option that has not been tested

on CUBEP3M yet, but which is, in principle at least, still available.

The standard package also contains support for gas cosmological evolution through a portable TVD-

MHD module (Pen et al., 2003) that scales up to thousands of cores as well (see Pang et al. (2010) and

footnote 4 in section 2.5), and a coupling interface with the radiative transfer code C2-ray (Mellema


et al., 2006). CUBEP3M is therefore one of the most competitive and versatile public N-body codes,

and has been involved in a number of scientific applications over the last few years, spanning the field

of weak lensing (Vafaei et al., 2010; Lu & Pen, 2008; Dore et al., 2009; Lu et al., 2010; Yu et al., 2012;

Harnois-Deraps et al., 2012b), BAO (Zhang et al., 2011; Ngan et al., 2012; Harnois-Deraps & Pen, 2012;

Harnois-Deraps et al., 2012c), formation of early cosmic structures (Iliev et al., 2008, 2010), possible

observations of dark stars (Zackrisson et al., 2010; Ilie et al., 2012) and reionization (Iliev et al., 2012;

Fernandez et al., 2012; Friedrich et al., 2011; Mao et al., 2012; Datta et al., 2012b,a). Continuous efforts

are being made to develop, extend and improve the code and each of its utilities, and we expect that

this will pave the way to an increasing number of science projects. Notably, the fact that the force

calculation is multi-layered makes the code extendible, and opens the possibility to run CUBEP3M on

hybrid CPU-GPU clusters. It is thus important for the community to have access to a paper that

describes the methodology, the accuracy and the performance of this public code.

Since CUBEP3M is not new, the existing accuracy and systematic tests were performed by different

groups, on different machines, and with different geometric and parallelization configurations. It is not

an ideal situation in which to quantify the performance, and each test must therefore be viewed as a

separate measurement that quantifies a specific aspect of the code. We have tried to keep to a minimum

the number of such different test runs, and although the detailed numbers vary with the problem size

and the machines, the general trends are rather universal.

Tests on constraints of the redshift step size and on improvements of the force calculations were

performed on the Sunnyvale beowulf cluster at the Canadian Institute for Theoretical Astrophysics

(CITA). Each node contains 2 Quad Core Intel(R) Xeon(R) E5310 1.60GHz processors, 4GB of RAM,

a 40 GB disk and 2 gigE network interfaces. These tests were performed on the same cluster, but with

different box sizes, starting redshifts and particle numbers. Hereafter we refer to these simulation sets

as the CITA configurations, and specify which parameters were used in each case. Some complimentary

runs were also performed on the SciNet GPC cluster (Loken et al., 2010), a system of IBM iDataPlex

DX360M2 nodes equipped with Intel Xeon E5540 cores running at 2.53GHz with 2GB of RAM per core.

For tests of the code accuracy, of the non-Gaussian initial conditions generator and of the run time halo

finder algorithm, we used a third simulation configuration series that was run at the Texas Advanced

Computing Centre (TACC) on Ranger, a SunBlade x6420 system with AMD x86 64 Opteron Quad Core

2.3 GHz ‘Barcelona’ processors and Infiniband networking. These RANGER4000 simulations evolved

40003 particles from z = 100 to z = 0 with a box side of 3.2h−1Gpc on 4000 cores.

The paper is structured as follow: section 2.3 reviews the structure and flow of the main code; section

2.4 describes how Poisson equation is solved on the two-mesh system; we then present in section 2.5 the


scaling of the code to very large problems, including much larger runs that were produced at various

high performance computing centres; section 2.6 discuss the accuracy and systematic effects of the code.

We then describe in section 2.7 the run-time halo finder, in section 2.8 various extensions to the default

configuration, and conclude afterwards.

2.3 Review of the Code Structure

An optimal large scale N-body code must address many challenges: minimize the memory footprint to

allow larger dynamical range, minimize the passing of information across computing nodes, reduce and

accelerate the memory accesses to the large scale arrays, make efficient use of high performance libraries

to speed up standard calculations like Fourier transforms, etc. In the realm of parallel programming,

high efficiency can be assessed when a high load is balanced across all processors most of the time. In

this section, we present the general strategies adopted to address these challenges1. We start with a

walkthrough the code flow, and briefly discuss some specific sections that depart from standard N-body

codes, while referring the reader to future sections for detailed discussions on selected topics.

As mentioned in the Introduction section, CUBEP3M is a FORTRAN90 N-body code that solves

Poisson’s equation on a two-level mesh, with sub-cell accuracy thanks to particle-particle interactions.

The code has extensions that departs from this basic scheme, and we shall come back to these later, but

for the moment, we adopt the standard configuration. The long range component of the gravity force

is solved on the coarse grid, and is global in the sense that the calculations require knowledge about

the full simulated volume. The short range force and the particle-particle interactions are computed in

parallel on a second level of cubical decomposition of the local volumes, the tiles. To make this possible,

the fine grid arrays are constructed such as to support parallel memory access. In practice, this is done

by adding an additional dimension to the relevant arrays, such that each CPU accesses a unique memory

location. The force matching between the two meshes is performed by introducing a cutoff length, rc,

in the definition of the two force kernels. The value of rc = 16 fine cells was found to balance the

communication overhead between processes and the accuracy of the match between the two meshes.

The computation of the short range force requires each tile to store the fine grid density of a region

that includes a buffer surface around the physical volume it is assigned. The thickness of this surface

must be larger than rc, and we find that a 24 cells deep buffer is a good compromise between memory

usage and accuracy. This fully compensates for the coarse mesh calculations, whose CIC interpolation

1Many originate directly from MPT and were preserved in CUBEP3M; those will be briefly mentioned, and we shall

refer the reader to the original PMFAST paper for greater details.


scheme reaches two coarse cells deep beyond the cutoff. When it comes to finding haloes at run time,

this buffer can create a problem, because a large object located close to the boundary can have a radius

larger than the buffer zone, in which case it would be truncated and be assigned a wrong mass, centre of

mass, etc. It could then be desirable to increase the buffer zone around each tile, at the cost of a loss of

memory dedicated to the actual physical volume, and the code is designed to allow for such a flexibility.

Three important processes that speed up the code were already present in PMFAST: 1) access to

particles is accelerated with the use of linked lists, 2) deletion of ‘ghost’ particles in buffer zones is done

at the same time as particles are passed to adjacent nodes, and 3) the global FFTs are performed with

a slab decomposition of the volumes via a special file transfer interface, designed specifically to preserve

a high processor load.

Because the coarse grid arrays require 43 times less memory per node, it does not contribute much

to the total memory requirement, and the bulk of the foot-print is concentrated in a handful of fine grid

arrays. Some of these are only required for intermediate steps of the calculations, hence it is possible to

hide therein many coarse grid arrays2. We present here the largest arrays used by the code:

1. xv stores the position and velocity of each particle

2. ll stores the linked-list that accelerate the access to particles in each coarse grid cell

3. send buf and recv buf store the particles to be MPI-passed to the neighbouring sub-volumes

4. rho f and cmplx rho f store the local fine grid density in real and Fourier space respectively

5. force f stores the force of gravity (short range only) along the three Cartesian directions

6. kern f stores the fine grid force kernel in the three directions

7. PID stores the unique particle identification tags as double integers.

8. send buf PID and recv buf PID store the ID to be MPI-passed to the neighbouring sub-volumes

The particle ID is a feature that can be switched off simply by removing a compilation flag, and allows

to optimize the code for higher resolution configurations.

In terms of memory, CUBEP3M can be configured such as the main footprint is dominated exclusively

by the particle xv array and the linked list, in which case it asymptotes to about 28 bytes per particles.

For a fixed problem size, this can be achieved by maximizing the number of tiles, thereby reducing the

size of the local fine mesh arrays. For comparison, Lean-GADGET-2 code uses 84 bytes per particles

2This memory recycling is done with ‘equivalence’ statements in FORTRAN90.


(Springel, 2009), which is about three times more. There is a limit at which we can do such a volume

breakdown, though, since the physical volume on each tile must be at least twice as large as the buffer,

i.e. ≥ 963 cells (see section 2.4). This allows a low rate of particle exchange across nodes; in the default

configuration, the send buf and recv buf arrays are much larger, in which case the memory per particle

goes up to 44 bytes. Enabling the particle ID adds at least 8 bytes per particle from the main array,

and as the MPI-communication buffers are allocated larger sizes, they contribute at most an additional

16 bytes as well. Hence a ‘lean’ version with the particle ID tags uses down to 36 bytes per particles.

In practice, minimizing the memory does not optimize the speed, as many CPU-intensive operations are

performed on each tile – the FFTW for instance. A higher number of cells per tile – typically 1763 –

with fewer tiles usually runs faster, in which case the other arrays in the above-mentioned list become

significantly larger, and the memory per particle is closer to 100 - 140 bytes.

The code flow is presented in Fig. 2.1 and 2.2. Before entering the main loop, the code starts with an

initialization stage, in which many declared variables are assigned default values, the redshift checkpoints

are read, the FFTW plans are created, and the MPI communicators are defined. The xv array is obtained

from the output of the initial conditions generator (see section 2.8.1), and the force kernels on both grids

are constructed by reading precomputed kernels that are then adjusted to the specific simulation size.

For clarity, all these operations are collected under the subroutine call initialize in Fig. 2.1, although

they are actually distinct calls in the code.

Each iteration of the main loop starts with the timestep subroutine, which proceeds to a deter-

mination of the redshift jump by comparing the step size constraints from each force components and

from the scale factor. The cosmic expansion is found by Taylor expanding Friedmann’s equation up to

the third order in the scale factor, and can accommodate constant or running equation of state of dark

energy. The force of gravity is then solved in the particle mesh subroutine, which first updates the

positions and velocities of the dark matter particles, exchange with neighbouring nodes those that have

exited to volume, creates a new linked list, then solve Poisson’s equation. This subroutine is conceptually

identical to that of PMFAST, with the exceptions that CUBEP3M decomposes the volume into cubes (as

opposed to slabs), and computes pp interactions as well. The loop over tiles and the particle exchange

are thus performed in three dimensions. When the short range and pp forces have been calculated on

all tiles, the code exits the parallel OPENMP loop and proceeds to the long range calculation. This

section of the code is also parallelized on many occasions, but, unfortunately, the current MPI-FFTW

libraries do not allow multi-threading. There is thus an inevitable loss of efficiency during each global


program cubep3m

call initialize

do

call timestep

call particle_mesh

if(checkpoint_step) then

call checkpoint

elseif(last_step)

exit

endif

enddo

call finalize

end program cubep3m

Figure 2.1: Overall structure of the code simplified for readability.

Fourier transforms, during which only the single core MPI process is active3. As in PMFAST, the particle

position and velocity updates are performed in a leap frog scheme (Hockney & Eastwood, 1981).

If the current redshift corresponds to one of the checkpoints, the code advances all particles to their

final location and writes them to file. These restart files begin with a small header that contains the local

number of particles, the current redshift and time step, and the constraints on the time step jump from

the previous iteration; the header is followed by the position and velocity of each particle. The particle

identification tags are written with a similar fashion in distinct files to simplify the post-processing

coding. This general I/O strategy allows for highly efficient MPI parallel read and write, by default in

binary format for compactness, and has been shown to scale well to large data sets. At this stage, the

performance depends only on the setup and efficiency of the parallel file system.

Similarly, the code can compute two-dimensional projections of the density field, halo catalogues

(see section 2.7 for details), and can compute the power spectrum on the coarse grid at run time. The

code exits the main loop when it has reached the final redshift, it then wraps up the FFTW plans and

clears the MPI communicators. We have collected those operations under the subroutine finalize for

concision.

Other constraints that need to be considered is that any decomposition geometry limits in some

3Other libraries such as P3DFFT (http://code.google.com/p/p3dfft/) currently permit an extra level of paralleliza-tion, and it is our plan to migrate to one of these in the near future.


subroutine particle_mesh

call update_position + apply random offset

call link_list

call particle_pass

!$omp parallel do

do tile = 1, tiles_node

call rho_f_ngp

call cmplx_rho_f

call kernel_multiply_f

call force_f

call update_velocity_f

if(pp = .true.) then

call link_list_pp

call force_pp

call update_velocity_pp

if(extended_pp = .true.) then

call link_list_pp_extended

call force_pp_extended

call update_velocity_pp_extended

endif

endif

end do

!$omp end parallel do

call rho_c_ngp

call cmplx_rho_c

call kernel_multiply_c

call force_c

call update_velocity_c

delete_buffers

end subroutine particle_mesh

Figure 2.2: Overall structure of the two-level mesh algorithm. We have included the section that concerns the standard

pp and the extended pp force calculation, to illustrate that they follow similar linked-list logic.


ways the permissible grid sizes, and that volumes evenly decomposed into cubic sub-sections – such as

CUBEP3M – require the number of MPI processes to be a perfect cube. Also, since the decomposition is

volumetric, as opposed to density dependent, it suffers from load imbalance whenever the particles are

not evenly distributed. However, large scale cosmology problems are very weakly affected by this effect,

which can generally be overcome by allocating a small amount of extra memory per node. As mentioned

in section 2.5, large runs can even be optimized by allocating less memory to start with, follow by a

restart with a configuration that allows more memory per node, if required.

2.4 Poisson Solver

This section reviews how Poisson’s equation is solved on a double-mesh configuration. Many parts of the

algorithm are identical to PMFAST, hence we refer the reader to section 2 of MPT for more details. In

CUBEP3M, the mass default assignment scheme are a ‘cloud-in-cell’ (CIC) interpolation for the coarse

grid, and a ‘nearest-grid-point’ (NGP) interpolation for the fine grid (Hockney & Eastwood, 1981). This

choice is motivated by the fact that the most straightforward way to implement a P3M algorithm on

a mesh is to have exactly zero mesh force inside a grid, which is only true for the NGP interpolation.

Although CIC generally has a smoother and more accurate force, the pp implementation enhances the

code resolution by almost an order of magnitude.

The code units inherit from (Trac & Pen, 2004) and are summarized here for completeness. The

comoving length of a fine grid cell is set to one, such that the unit length in simulation unit is

1L = aL

N(2.1)

where a is the scale factor, N is the total number of cells along one dimension, and L is the comoving

volume in h−1Mpc. The mean comoving mass density is also set to unity in simulation units, which, in

physical units, corresponds to

1D = ρm(0)a−3 = Ωmρca−3 =

3ΩmH2o

8πGa3(2.2)

Ωm is the matter density, Ho is the Hubble’s constant, ρc is the critical density today, and G is Newton’s

constant. The mass unit is found with M = DL3. Specifying the value of G on the grid fixes the time

unit, and with Ggrid = 1/(6πa), we get:

1T =2a2

3

1√ΩmH2

o

(2.3)

These choices completely determine the convertion between physical and simulation units. For instance,

the velocity units are given by 1V = L/T .


The force of gravity on a mesh can be computed either with a gravitational potential kernel ωφ(x)

or a force kernel ωF (x). Gravity fields are curl-free, which allows us to relate the potential φ(x) to the

source term via Poisson’s equation:

∇2φ(x) = 4πGρ(x) (2.4)

We solve this equation in Fourier space, where we write:

φ(k) =−4πGρ(k)

k2≡ ωφ(k)ρ(k) (2.5)

The potential in real space is then obtained with an inverse Fourier transform, and the kernel becomes

ωφ(x) = −G/r, with r = |x|. Using the convolution theorem, we can write

φ(x) =

∫ρ(x′)ωφ(x

′ − x)dx′ (2.6)

and

F(x) = −m∇φ(x) (2.7)

Although this approach is fast, it involves a finite differentiation at the final step, which enhances the

numerical noise. We therefore opt for a force kernel, which is more accurate, even though it requires four

extra Fourier transforms. In this case, we must solve the convolution in three dimensions and define the

force kernel ωF such as:

F(x) =

∫ρ(x′)ωF (x

′ − x)dx′ (2.8)

Because the gradient that is acting on the potential affects only unprime variables in the right side of

Eq. 2.6, we can express the force kernel as a gradient of the potential kernel. Namely:

ωF (x) ≡ −∇ωφ(x) = −mGrr2

(2.9)

Following the spherically symmetric matching technique of MPT (section 2.1), we split the force

kernel into two components, for the short and long range respectively, and match the overlapping region

with a polynomial. Namely, we have:

ωs(r) =

ωF (r) − β(r) if r ≤ rc

0 otherwise

(2.10)

and

ωl(r) =

β(r) if r ≤ rc

ωF (r) otherwise

(2.11)


The vector β(r) is related to the fourth order polynomial that is used in the potential case described in

MPT by β = −∇α(r). The coefficients are found by matching the boundary conditions at rc up to the

second derivative, and we get

β(r) =

[− 7r

4r3c+

3r3

4r5c

]r (2.12)

Since these calculations are performed on two grids of different resolution, a sampling window function

must be convoluted both with the density and the kernel (see [Eq. 7-8] of MPT). When matching the

two force kernels, the long range force is always on the low side close to the cutoff region, whereas

the short range force is uniformly scattered across the theoretical 1/r2 value – intrinsic features of the

CIC and NGP interpolation schemes respectively. By performing force measurements on two particles

randomly placed in the volume, we identified a small region surrounding the cutoff length in which

we empirically adjust both kernels such as to improve the match. Namely, for 14 ≤ r ≤ 16 in fine

cell units, ωs(r) → 0.985ωs(r), and for 12 ≤ r ≤ 16, ωl(r) → 1.2ωl(r). A recent adjustment of the

similar kind was also applied on the fine kernel to compensate for a systematic underestimate of the

force at fine grid distances. This is caused by the uneven balance of the force about the 1/r2 law in the

NGP interpolation scheme. Although most figures in this paper were made prior this correction, the

new code configuration applies a 90 per cent boost to the kernel for the six elements that are adjacent,

i.e. ωs(r = 1) → 1.9ωs(r = 1) (see section 2.6.2 for more details). As mentioned in section 2.3 and

summarized in Fig. 2.2, the force kernels are first read from files in the code initialization stage. Eq.

2.8 is then solved with fast Fourier transforms along each direction, and is applied onto particles in the

update velocity subroutine.

The pp force is calculated during the fine mesh velocity update, which avoids loading the particle list

twice and allows the operation to be threaded without significant additional work. During this process,

the particles within a given fine mesh tile are first read in via the linked list, then their velocity is

updated with the fine mesh force component, according to their location within the tile. In order to

organize the particle-particle interactions, we proceed by constructing a set of threaded fine-cell linked list

chains for each coarse cell. We then calculate the pairwise force between all particles that lie within the

same fine mesh cell, excluding pairs whose separation is smaller than a softening length rsoft; particles

separated by less than this distance thus have their particle-particle force set to zero. As this proceeds,

we accumulate the pp force applied on each particle and then determine the maximum force element of

the pp contribution, which is also taken into account when constraining the length of the global time

step.

Force softening is generally required by any code to prevent large scattering as r → 0 that can other-


wise slow the calculation down, and to reduce the two-body relaxation, which can affect the numerical

convergence. Many force softening schemes can be found in the literature, including Plummer force, uni-

form or linear density profiles or the spline-softened model (see Dyer & Ip (1993) for a review). In the

current case, a sharp force cutoff corresponds to a particle interacting with a hollow shell. In comparison

with other techniques, this force softening is the easiest to code and the fastest to execute. Generally,

it is desirable to match the smoothness of the force to the order of the time integration. A Plummer

force is infinitely differentiable, which is a sufficient but not necessary condition for our 2nd order time

integration. Also, one of the drawbacks of Plummer’s softening is that the resolution degrades smoothly:

the effects of the smoothing are present at all radii. In comparison, the uniform density and hollow shell

alternatives both have the advantage that the deviations from 1/r2 are minimized. Although all the

results presented in this paper were obtained with the sharp cutoff softening, other schemes can easily

be adopted as these are typically single line changes to the code. Dyer & Ip (1993) argues in favour of

the uniform density profile scheme – which is more physical – and future developments of CUBEP3M

will incorporate this option.

The choice of softening length is motivated by a trade off between accuracy and run time. Larger

values reduce the structure formation but make the code run quicker. We show in Fig. 2.3 the impact

of changing this parameter on the power spectrum. For this test, we produced a series of SciNet256

simulations, each starting at z = 100 and evolving to z = 0.5, with a box size of 100h−1Mpc. They each

read the same set of initial conditions and random seeds, such that the only difference between them is

the softening length. We record in Table 2.1 the real time for each trials, from where we witness the

strong effect of the choice of rsoft on the computation time. In this test, which is purposefully probing

rather deep in the non-linear regime, reducing the softening length to half its default value of 1/10th of a

fine grid cell doubles the run time, while the gain in power spectrum is less than two per cent. Similarly,

increasing rsoft to 0.2 reduces the run time almost by a half, but suffers from a five per cent loss in

power at the resolution turn-around. One tenth of grid cell seems to be the optimal choice in this trade

off, however it should really be considered a free parameter.

In addition, PMFAST could run with a different set of force kernels, described in MPT as ‘least square

matching’, a technique which adjusts the kernels on a cell-by-cell basis such as to minimize the deviation

with respect to Newtonian predictions. This was originally computed in the case where both grids are

obtained from CIC interpolation. Moving to a mix CIC/NGP scheme requires solving the system of

equations with the new configuration, a straightforward operation. With the inclusion of the random

shifting (see section 2.6.2), however, it is not clear how much improvement one would recover from this

other kernel matching. It is certainly something we will investigate and document in the near future.


10−1

100

101

10−1

100

101

102

k[h/Mpc]

∆2(k

)

0.050.10.20.30.5

101

101.8

101.9

Figure 2.3: Dark matter power spectrum, measured at z = 0.5 in a series of SciNet256 simulations in which the softening

length is varied. The simulations started off at z = 100, and have a box size of 100h−1Mpc.

Table 2.1: Scaling in CPU resources as a function of the softening length, in units of fine grid cells.

rsoft time (h)

0.5 5.75

0.3 8.17

0.2 10.09

0.1 18.67

0.05 31.07


Figure 2.4: Scaling of CUBEP3M on Curie fat nodes (left) and on Ranger TACC facility for very large number of cores

(right). Plotted is the code speedup (N3particles/twallclock) against core count, normalized by the smallest run in each case.

The dashed line indicates the ideal weak scaling, and details on the data are listed in Table 2.2.

Finally, a choice must be done concerning the longest range of the coarse mesh force. Gravity can be

either a) a 1/r2 force as far as the volume allows, or b) modified to correctly match the periodicity of

the boundary conditions. By default, the code is configured along to the second choice, which accurately

models the growth of structures at very large scales. However, detailed studies of gravitational collapse

of a single large object would benefit from the first setting, even though the code is not meant to evolve

such systems as they generally require load balance control.

2.5 Scaling Performances

The parallel algorithm of CUBEP3M is designed for ‘weak’ scaling, i.e. if the number of cores and

the problem size increase in proportion to each other, then for ideal scaling the wall-clock time should

remain the same. This is to be in contrasted with ‘strong’ scaling codes, whereby the same problem

solved on more cores should take proportionately less wall-clock time. This weak scaling requirement is

dictated by the problems we are typically investigating (very large and computationally-intensive) and

our goals, which are to address such large problems in the most efficient way, rather than for the least

wall-clock time. Furthermore, we recall that there is no explicit load balancing feature, thus the code is

maximally efficient when the sub-domains contain roughly an equal number of particles. This is true for

most cosmological-size volumes that do not resolve too deep in the non-linear regime, but not for e.g.


Table 2.2: Scaling of CUBEP3M on Curie. Speedup is scaled to the smallest run.

number of cores speedup ideal speedup absolute timing (min) Nparticles box size (h−1Mpc)

32 1.00 - 3.2 2563 256

256 7.21 8 3.55 5123 512

864 25.63 27 4.8 8643 864

2048 61.87 64 26.48 20483 2048

Table 2.3: Scaling of CUBEP3M on Ranger. Speedup is scaled to the smallest run.

number of cores speedup ideal speedup absolute timing (min) Nparticles box size (h−1Mpc)

864 1.00 - 258 17283 6.3

4096 4.53 4.74 320 30723 11.4

10976 9.78 12.7 845 54883 20

21952 14.73 25.4 561 54883 20

simulations of a single highly-resolved galaxy.

We first recall that because of the volumetric decomposition, the total number of MPI processes

needs to be a perfect cube. Also, for maximal resource usage, the number of tiles per node should be a

multiple of the number of available CPUs per MPI process, such that no core sits idle in the threaded

block. Given the available freedom in the parallel configuration, as long as the load is balanced, it is

generally good practice to maximize the number of OPENMP threads and minimize the number of MPI

processes: the information exchange between cores that are part of the same motherboard is generally

much faster. In addition, having fewer MPI processes reduces the total amount of buffer zones, freeing

memory that can be used to increase the mesh resolution. However, it has been observed that for the

case of non-uniform memory access (NUMA) systems, the code is optimized when only the cores that

share the same socket are OPENMP threaded. As one probes deeper into the non-linear regime however,

the formation of dense objects can cause memory problems in such configurations, and increasing the

number of MPI processes helps to ensure memory locality, especially in NUMA environments.

The intermediate version of the code – CUBEPM – was ported to the IBM Blue Gene/L platform,

and achieved weak-scaling up to 4096 processes (over a billion particles), with the N-body calculation

incurring only a 10 per cent overhead at runtime (compared to 8 processes) for a balanced workload4. In

order to accommodate the limited amount of memory available per processing core on the Blue Gene/L

platform machines, it was necessary to perform the long range MPI FFT with a volumetric decomposition

4http://web.archive.org/web/20060925132146/ http://www-03.ibm.com/servers/deepcomputing/pdf/

Blue Gene Applications Paper CubePM 032306.pdf


(Eleftheriou et al., 2005). Slab decomposition would have required a volume too large to fit in system

memory given the constraints in the simulation geometry.

The scaling of CUBEP3M was first tested with a dedicated series of simulations – the CURIE simula-

tion suite– by increasing the size and number of cores on the ‘fat’ (i.e. large-memory) nodes of the Curie

supercomputer at the Tres Grand Centre de Calcul (TGCC) in France. For appropriate direct compar-

ison, all these simulations were performed using the same particle mass (Mparticle = 1.07 × 1011M⊙)

and force resolution (softening length 50 h−1kpc). The box sizes used range from 256 h−1Mpc to 2048

h−1Mpc, and the number of particles from 2563 to 20483. Simulations were run on 32 up to 2048 com-

puting cores, also starting from redshift z = 100, and evolving until z = 0. Our results are shown in Fig.

2.4 and in Table 2.2, and present excellent scaling, within ∼ 3 per cent of ideal, at least for up to 2048

cores.

We have also ran CUBEP3M on a much larger number of cores, from 8000 to up to 21,976, with 54883-

60003 (165 to 216 billion) particles on Ranger and on JUROPA at the Julich Supercomputing Centre in

Germany, which is an Intel Xeon X5570 (Nehalem-EP) quad-core 2.93 GHz system, also interconnected

with Infiniband. Since it is not practical to perform dedicated scaling tests on such a large number of

computing cores, we instead list in Table 2.3 the data directly extracted from production runs. We have

found the code to scale within 1.5 per cent of ideal up to 4096 cores. For larger sizes (≥10,976 cores), the

scaling is less ideal, due to increased communication costs, I/O overheads (a single time slice of 54883

particles is 3.6 TB) and load balancing issues, but still within ∼ 20 per cent of ideal. These first three

Ranger runs were performed with 4 MPI processes and 4 threads per Ranger node (‘4way’)5.

Furthermore, due to the increasing clustering of structures at those small scales, some of the cuboid

sub-domains came to contain a number of particles well above the average, thereby requiring more

memory per MPI process in order to run until the end. As a consequence, throughout most of their late

evolution, the largest two of these simulations were run with 4096 and 21,952 cores and with only 2 MPI

processes and 8 threads per node (‘2way’), which on Ranger allows using up to 16 GB of RAM per MPI

process6. Because each processor accesses memory that is not fully local, this configuration does affect

the performance somewhat, as does the imperfect load balancing that arises in such situations. This can

be seen in the rightmost point of Fig. 2.4 (right panel), where the scaling is 42 per cent below ideal.

We note that we still get ∼ 1.5 speedup from doubling the core count, even given these issues. Overall

5For these very large runs, we used a NUMA script tacc affinity, specially provided by the technical staff, that bindthe memory usage to local sockets, thus ensuring memory affinity. This becomes important because the memory socketsper node (32 GB RAM/node on Ranger) are actually not equal-access. Generally, the local memory of each processor hasmuch shorter access time.

6In order to ensure local memory affinity, a second special NUMA control script, tacc affinity 2way, was developed forus by the TACC technical staff and allowed to run more efficiently in this mode.


the code scaling performance is thus satisfactory even at extremely large number of cores. We expect

the code to handle even larger problems efficiently, and is thus well suited to run on next generation

Petascale systems.

Finally, we note that several special fixes had to be developed by the TACC and JUROPA technical

staff in order for our largest runs to work properly. In particular, we encountered unexpected problems

from software libraries such as MPICH and FFTW when applied to calculations of such unprecedented

size.

2.6 Accuracy and Systematics

This section describes the systematics effects that are inherent to our P3M algorithm. We start with

a demonstration of the accuracy with a power spectrum measurement of a RANGER4000 simulation.

The halo mass function also assess the code capacity to model gravitational collapse, but depends on

the particular halo finder used. We thus postpone the discussion on this aspect until section 2.7, and

focus for the moment on the particles only.

We then quantify the accuracy of the force calculation with comparisons to Newton’s law of gravita-

tion. Since most of the systematics come from the contribution at smaller distances, we dedicate a large

part of this section to the force calculation at the grid scale. We then explain how the correction on the

fine force kernel at r = 1 was obtained, and present how constraining the size of the redshift jumps can

help improve the accuracy of the code.

2.6.1 Density and power spectrum

One of the most reliable ways to assess the simulation’s accuracy at evolving particles is to measure the

density power spectrum at late time, and compare to non-linear prediction. For an over-density field

δ(x ), the power spectrum is extracted from the two point function in Fourier space as:

〈|δ(k)δ(k ′)|〉 = (2π)3P (k)δD(k ′ − k) (2.13)

where the angle bracket corresponds to an ensemble (or volume) average. Fig. 2.5 presents a 2-

dimensional density projection of a RANGER4000 simulation, which evolved 40003 particles until redshift

zero. We then plot in Fig. 2.6 its dimensionless power spectrum, defined as ∆2(k) = k3P (k)/(2π2),

and observe that the agreement with the non-linear prediction of Smith et al. (2003) is within five per

cent up to k = 1.0hMpc−1. The code exhibits a ∼ 5 per cent over estimate compared to the theory

for 0.2 < k < 0.6hMpc−1, an effect generally caused by inaccuracies at the early time steps. It can be


removed by starting the code at a later redshift, but then the code has less time to relax, which reduces

the accuracy on other quantities like the halo mass functions or the four point functions. The drop of

power at higher k is partly caused by the finite mesh resolution, partly from expected deviations about

the predictions. The fluctuations at low-k come from the white noise imposed in the initial conditions,

and it was shown in Ngan et al. (2012) and Harnois-Deraps & Pen (2012) that samples of a few hundreds

of realizations average to the correct value.

2.6.2 Mesh force at grid distances

The pairwise force test presented in MPT was carried by placing two particles at random locations on

the grid, calculating the force between them, then iterating over different locations. This test is useful

to quantify the accuracy on a cell-by-cell basis, but lacks the statistics that occur in a real time step

calculation. The actual force of gravity in the P3M algorithm, as felt by a single particle during a single

step, is presented in the top left panel of Fig. 2.7. This force versus distance plot was obtained from a

CITA128 realization, and the calculation proceeds in two steps: 1- we compute the force on each particle

in a given time step. 2- we remove a selected particle, thus creating a ‘hole’, compute the force again on

all particles, and record on file the force difference (before and after the removal) as a function of the

distance to the hole.

Particles in the same fine cell as the hole follow the exact 1/r2 curve. The scatter at distances of

the order of the fine grid is caused by the NGP interpolation scheme: particles in adjacent fine cells can

be actually very close, as seen in the upper left region of this plot, but still feel the same mesh force at

grid cell distances. This creates a discrepancy up to an order of magnitude, in loss or gain, depending

on the location of the pair with respect to the centre of the cell. As the separation approaches a tenth

of the full box size or so, the force on the coarse mesh scatters significantly about Newton’s law due to

periodic boundary conditions. As mentioned at the end of section 2.4, the longest range of force kernel

can either model accurately Newtonian gravity, or the structure growth of the largest modes, but not

both. For the sake of demonstrating the accuracy of the force calculation, we chose the first option in

this section, but this is not how the code would normally be used.

The top right panel of Fig. 2.7 shows the fractional error on the force along the radial direction (top)

and the fractional tangential contribution (bottom), also calculated from a single time step. Again,

particles that are in the same cell have zero fractional error in the radial direction, and zero tangential

force. Particles feel a large scatter from the neighbouring fine mesh cells, but as the distance increase

beyond five fine cells, the fractional error drops down to about 20 per cent. The transverse fractional


Figure 2.5: Detail of a dark matter density 3.2h−1Gpc per side, projected over 15h−1Mpc and measured

at z = 0 in a RANGER4000 simulation. In the online version, particles are shown in pink, haloes in

blue, and the colour scale is linear. The full projection is 64 times larger, but we show here only a

sub-volume, which provides more details on the structure.


10−2

10−1

100

10−10

10−5

100

∆2(k

)

CUBEP3MCAMB

10−2

10−1

100

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

k[h/Mpc]

∆P

/P

Figure 2.6: Dark matter power spectrum, measured at z = 0 in a RANGER4000 simulation, compared to the non-linear

predictions of HALOFIT (calculated with the online CAMB toolkit). The vertical line corresponds to the scale of the

coarse mesh, while the dotted line represents the Poisson noise. The particle masses are of 5.68× 1010M⊙.


error peaks exactly at the grid distance, and is everywhere about 50 per cent smaller than the radial

counterpart.

Although these scatter plots show the contributions from individual particles and cells, it is not clear

whether the mean radial force felt is higher or lower than the predictions. For this purpose, we rebin

the results in 50 logarithmically spaced bins and compute the mean and standard deviation. We show

the resulting measurement in the middle left panel of Fig. 2.7, where we observe that on average, there

is a 50 per cent fractional error in the range 0.4 < r < 2, but the numerical calculations are otherwise in

excellent agreement with Newton’s law. Since the transverse force is, by definition, a positive number,

and since we know that it averages out to zero over many time steps, we plot only the scatter about

the mean, represented in the figure by the solid line. The transverse scatter is smaller everywhere for

r < 10.

At grid scales, the fractional error is larger than PMFAST, largely due to the fact that the fine

mesh force is performed with an NGP interpolation scheme – as opposed to CIC. This prescription is

responsible for the larger scatter about the theoretical value, but, as mentioned earlier, NGP interpolation

is essential to our implementation of the pp part. At the same time, the biggest problem with the

straightforward pp force calculation is that the results are anisotropic and depend on the location of the

fine mesh with respect to the particles. As an example, consider two particles on either side of a grid

cell boundary, experiencing their mutual gravity attraction via the fine mesh force with a discretized

one-grid cell separation. If, instead, the mesh was shifted such that they were within the same cell,

they would experience the much larger pp force. This is clearly seen in the top left panel of Fig. 2.7,

where particles physically close, but in different grid cells, feel a force up to an order of magnitude too

small. This effect is especially pronounced at the early stages of the simulation where the density is

more homogeneous, and leads to mesh artefacts appearing in the density field.

In order to minimize these two systematic effects – the large scatter and the anisotropy – we ran-

domly shift the particle distribution relative to the mesh by a small amount – up to 2 fine grid cells in

magnitude – in each dimension and at each time step. This adds negligible computational overhead as it

is applied during the particle position update, and suppresses the mesh behaviour that otherwise grows

over multiple time steps. It is possible to shift back the particles at the end of each time step, which

prevents a random drift of the whole population, a necessary step if one needs to correlate the initial

and final positions of the particles for instance, or for hybrid dark matter – MHD simulations.

We ensure that, on average, this solution balances out the mesh feature, by tuning the force kernels

such as to provide a force as evenly balanced as possible, both at the cutoff length (rc = 16 fine cells)

and at grid cell distances (see discussion in section 2.4). Both of these adjustments are performed from


100 101 102

−0.5

0

0.5

1

1.5

2

r[cells]

∆F

/F

radialtangential

100 101 102

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

r[cells]

∆F

/F

radialtangential

Figure 2.7: (top left:) Gravity force in the P3M algorithm, versus distance, compared with the exact 1/r2 law. Distance

is in fine mesh cell units, and force in simulation units. This particular calculation was obtained in a CITA128 realization

with a box size of 100h−1Mpc, in a single time step. (top right:) Fractional error on the force in the radial direction

(top) and fractional tangential contribution (bottom). In a full simulation run, the scatter averages over many time steps,

thanks to the inclusion of a random offset that is imposed on each particle, as discussed in the main text. (middle row:)

Top row organized in 50 logarithmic bins; for the tangential contribution, we plot only the scatter about the mean (solid

line). (bottom row:) Same as middle row, but averaged over ten time steps.


10−2

10−1

100

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

k[h/Mpc]

∆2(k

)

Figure 2.8: Dark matter power spectrum, measured at z = 180, 100, 20 and 10, in a CITA256 realization that is 1000h−1

Mpc per side. The dashed line represents the initial condition power spectrum, the dotted lines are the linear predictions,

and the open circles the standard P3M configuration. The dots are obtained by simply removing the random offset that

is normally applied at each time step.

the ‘hole’ and the pairwise force tests mentioned above. The bottom panels of Fig. 2.7 show the effect

of averaging the force calculation over ten time steps; the random offset is applied on the particles (and

on the hole) at the end of each force calculation. We observe that the agreement with Newton’s law is

now at the per cent level for r ≥ 1.5. At smaller distances, we still observe a scatter, but it is about two

times smaller than during a single time step for r > 1. The force at sub-grid distances is, on average,

biased on the low side by 20-40 per cent, as caused by the discretization effect of the NGP.

We note that this inevitable loss of force in the NGP scheme is one of the driving arguments to

extend the pp calculation outside the fine mesh cell, since the scattering about the actual 1/r2 law drops

rapidly with the distance. As discussed in section 2.8.3, this gain in accuracy comes at a computational

price, but at least we have the option to run the code in a higher precision mode.

We present in Fig. 2.8 the dramatic impact of removing the random offset in the otherwise default

code configuration. This test was performed with a CITA256 simulation of very large box size, the

output redshifts are very early (the upper most curve at z = 10 was obtained after only 60 time steps),

such that the agreement with linear theory should extend up to the resolution limit. Instead, we observe

that the power spectrum grows completely wrong, due to the large scatter in the force from the fine

mesh, and to the anisotropic nature of the P3M calculations mentioned above. When these effects are

not averaged over, the errors directly add up at each time step, which explains why later times are worst.

We recall that PMFAST did not have this problem since it used CIC interpolation on both meshes.


As mentioned in section 2.4, most of the tests presented in this paper were obtained without the fine

force kernel correction at r = 1. For completeness, we present here the motivation behind the recent

improvement.Without the correction, the force of gravity looks like Fig. 2.9, where we observe that the

fractional error in the radial direction is systematically on the low side, by up to 50 per cent, even after

averaging over 10 time steps. This is caused by the fact that in the strong regime of a 1/r2 law, the

mean force from a cell of uniform density, as felt by a particle, is not equivalent to that of the same

cell collapsed to its centre – the actual position of equivalence would be at a closer distance. Basically,

the idea is to boost the small distance elements of the NGP kernel by a small amount, which thereby

corrects for the loss of force observed in Fig. 2.9.

We show in Fig. 2.10 the impact on the power spectrum of three different correction trials: a) the

force from the first layer of neighbours is boosted by 90 per cent, b) the force from the first two layers

of neighbours is boosted by 60 per cent, and c) the force from two layers is boosted by 80 per cent. We

observe that with the trial a), we are able to improve the power at turn around by a factor of 1.5, which

almost doubles the resolution. This test was performed in a series of SciNet256 simulations with a box

length of 150h−1Mpc evolved until z = 0. We see from the figure that in the original configuration,

the departure from the non-linear predictions occur at k ∼ 3.0hMpc−1, whereas the first trial allows for

a similar resolution down to k ∼ 4.5hMpc−1. The two other correction schemes are more aggressive,

and although they recover power at even smaller scales, they exhibit a small overestimate compared to

non-linear predictions in the range 2.0 < k < 4.0hMpc−1, and more testing should be done prior to using

these corrections. We have also tried to increase even more the boost factor in the case a) but once again,

the boost started to propagate to larger scales, an effect we need to avoid. We therefore conclude that

the fine force kernel should be subject to a boost factor following ‘trial a’, i.e. ωs(r = 1) → 1.9ωs(r = 1).

It is thinkable that other corrections could outperform this one, hence the kernel correction factor

should also be considered as an adjustable parameter: ‘no correction’ is considered as the conservative

mode, which is accurate only up to the turn around observed in the dimensionless power spectrum, and

which has been used in most of the tests presented in this paper. Since higher resolution can be achieved

at no cost with the corrected fine force kernel, it has now been adopted in the default configuration; ‘trial

b’ and ‘trial c’ should be considered as too aggressive. We note, again, that we have not exhausted the

list of possible correction, and that future work in that direction might result in even better resolution

gain.


100

101

102

−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

r[cells]

∆F

/F

radialtangential

Figure 2.9: Fractional error in the gravity force, without the fine force kernel correction at r = 1. This particular

calculation was obtained in a CITA128 realization with a box size of 150h−1Mpc, averaged over ten time steps.

10−1

100

101

10−1

100

101

102

k[h/Mpc]

∆2(k

)

DefaultabcCAMB

100.2

100.9

102

Figure 2.10: Dark matter power spectrum, measured at z = 0, in a SciNet256 realization that is 150h−1 Mpc per

side. The thick solid line represents predictions from CAMB, the thin solid line is the default (i.e. without correction)

configuration of the code, and the three other lines represent three different corrections that are applied at grid distances

to compensate for the systematic force underestimation seen in the bottom panels of Fig. 2.7.


2.6.3 Constraining redshift jumps

At early stages of the simulation, the density field is homogenous, causing the force of gravity to be rather

weak everywhere. In that case, the size of the redshift jumps is controlled by a limit in the cosmological

expansion. If the expansion jump is too large, the size of the residual errors can become significant,

and one can observe, for instance, a growth of structure that does not match the predictions of linear

theory even at the largest scales. One therefore needs to choose a maximum step size. In CUBEP3M,

this is controlled by rmax, which is the fractional step size, da/(a + da) and is set to 0.05 by default.

Generally, a simulation should start at a redshift high enough so that the initial dimensionless power

spectrum is well under unity at all scales. This ensures that the Zel’dovich approximation holds at the

per cent level at least. Otherwise, truncation error at the early time steps will be significant, causing a

drop of accuracy.

It is possible to reduce this effect, and thereby improve significantly the accuracy of the code, by

modifying the value of rmax, at the cost of increasing the total number of time steps. Fig. 2.11 shows a

comparison of late time power spectra of a series of CITA256 realizations that originate from the same

initial conditions, and used the same random seeds to control the fine mesh shifts (mentioned above):

only the value of rmax was modified between each run. We observe that the impact is mostly located in

the non-linear regime, where decreasing the time step to 0.006 allows the simulation to recover about 30

per cent of dimensionless power at the turn over scale. This gain is greatly dependent on the choice of

initial redshift, the resolution and the box size, and ideally one would make test runs in order to optimize

a given configuration. As expected, the CPU resources required to run these simulations increase rapidly

as rmax decreases, as seen in Table 2.4. In this test case, reducing further at 0.001 shows only a mild ∼ 5

per cent improvement in accuracy over 0.006, but the increase in time is more than a factor of four. The

default configuration of the code is set to rmax = 0.05, but in the light of this recent test, we recommend

reducing to the value 0.01 or even 0.005. We also mention here that with a proper use of second order

initial conditions, it is possible to start the simulations at much later redshifts without loosing much

accuracy.

2.7 Runtime Halo Finder

We have implemented a runtime halo finding procedure, which we have developed based on the spherical

overdensity (SO) approach (Lacey & Cole, 1994). In the interest of speed and efficiency, the halo

catalogues are constructed on-the-fly at a pre-determined list of redshifts. The halo finding is massively-

parallel and threaded based on the main CUBEP3M data structures discussed in section 2.3. The code


10−1

100

10−2

10−1

100

101

102

k[h/Mpc]

∆2(k

)

0.0010.0060.06CAMB

100

100.3

101.4

101.7

Figure 2.11: Dark matter power spectrum, measured at z = 0 in a series of CITA256 realizations. The starting redshift

was raised to z = 200 to enhance the systematic effect. The different curves show different values of rmax. The resources

required to run these simulations increase rapidly as rmax decreases, as seen in Table 2.4.

Table 2.4: Scaling in CPU resources as a function of the value of rmax. The tests were performed on the CITA Sunnyvale

cluster, and general trends could vary slightly on other machines. The time tabulated on the right column corresponds to

the full run time of the CITA256 simulations, which evolved 1283 particles from z = 200 down to z = 0.

rmax time (h)

0.1 1.46

0.06 1.48

0.01 1.67

0.006 1.91

0.003 2.83

0.002 4.13

0.001 8.15


first builds the fine-mesh density for each sub-domain using CIC or NGP interpolation. It then proceeds

to search for all local density maxima above a certain threshold (typically set to a factor of 100 above

mean density) within each local tile. It then uses parabolic interpolation on the density field to determine

more precisely the location of the maximum within the densest cell, and records the peak position and

value.

Once the list of peak positions is generated, they are sorted from the highest to the lowest density.

Each halo candidates is then inspected independently, starting with the highest peak. The grid mass

is accumulated in spherical shells of fine grid cells surrounding the maximum, until the mean density

within the halo drops below a pre-defined overdensity cutoff (usually set to 178 in units of the mean,

in accordance with the top-hat collapse model). As we accumulate the mass, we remove it from the

mesh, so that no element is double-counted. This method is thus inappropriate for finding sub-haloes

since, within this framework, those are naturally incorporated in their host haloes. Because the haloes

are found on a discrete grid, it is possible, especially for those with lower mass, to overshoot the target

overdensity. To minimize this effect, we correct the halo mass and radius with an analytical density

profile. We use the Truncated Isothermal Sphere (TIS) profile (Shapiro et al., 1999; Iliev & Shapiro,

2001) for overdensities below ∼ 130, and a simple 1/r2 law for lower overdensities. TIS yields a similar

outer slope to the Navarro, Frenk and White (NFW Navarro et al., 1997) profile, but extends to lower

overdensities and matches well the virialization shock position given by the self-similar collapse solution

of Bertschinger (1985).

After the correct halo mass, radius and position are determined, we find all particles that are within

the halo radius. Their positions and velocities are used to calculate the halo centre-of-mass, bulk velocity,

internal velocity dispersion and the three angular momentum components, all of which are then included

in the final halo catalogues. We also calculate the total mass of all particles within the halo radius, also

listed in the halo data. This mass is very close, but typically slightly lower, than the halo mass calculated

based on the gridded density field. The centre-of-mass found this way closely follows that found from

the peak location, which is based on the gridded mass distribution.

A sample halo mass function produced from a RANGER4000 simulation at z = 0 is shown in Fig.

2.12. We compare our result to the precise fit presented recently by Tinker et al. (2008). Unlike most

other widely-used fits like the one by Sheth & Tormen (2002), which are based on friends-of-friends

(FOF) halo finders, this one relies on the SO search algorithm, whose masses are systematically different

from the FOF masses (e.g. Reed et al., 2007; Tinker et al., 2008), making this fit a better base for

comparison here. Results show excellent agreement, within ∼ 10 per cent for all haloes with masses

corresponding to 1000 particles or more. Lower-mass haloes are somewhat under-counted compared to


Figure 2.12: Simulated halo multiplicity function, M2

ρdndM

, based on a RANGER4000 simulation with 3.2h−1Gpc box

and 40003 particles (solid, red in the online version). For reference we also show a widely-used fit by Tinker et al. (2008)

(blue, dotted). The particle masses are of 5.68× 1010M⊙.

the Tinker et al. (2008) fit, by ∼ 20 per cent for 400 particles and by ∼ 40 per cent for 50 particles.

This is largely due to the grid-based nature of our SO halo finder, which misses some of the low-mass

haloes. It was shown that using more sophisticated halo finders (available only through post-processing

calculations due to their heavier memory footprint) it is possible to recover most of the expected number

count.

2.8 Beyond the standard configuration

Whereas all the preceding sections contain descriptions and discussions that apply to the standard

configuration of the code, many extensions have been recently developed in order to enlarge the range

of applications of CUBEP3M; this section briefly describes the most important of these improvements.

2.8.1 Initial conditions

As mention in section 2.3, the code starts off by reading a set of initial conditions. These are organized

as a set of 6 × N phase-space arrays – one per MPI process – where N is the number of particles in

the local volume. Each MPI sub-volume generates its own random seed, which is used to create a local

noise map. The initial power spectrum is found by reading off a transfer function at redshift z = 0 that


is generated from CAMB by default, and evolved to the desired initial redshift with the linear ΛCDM

growth function. The noise map is then Fourier transformed, each element is multiplied by the (square

root of the) power spectrum, and the result is brought back to real space. To ensure that the global

volume is correct, a buffer layer in the gravitational potential is exchanged across each adjacent node.

The particles are then moved from a uniform cell-centred position to a displaced position with Zel’dovich

approximation7.

The density field produced by this algorithm follows Gaussian statistics, and is well suited to describe

many systems. To increase the range of applicability of this code, we have extended this Gaussian initial

conditions generator to include non-Gaussian features of the ‘local’ form, Φ(x) = φ(x) + fNLφ(x)2 +

gNLφ(x)3, where φ(x) is the Gaussian contribution to the Bardeen potential Φ(x) (see Bartolo et al.

(2004) for a review). We adopted the CMB convention, in which Φ is calculated immediately after the

matter- radiation equality (and not at redshift z = 0 as in the large scale structure convention). For

consistency, φ(x) is normalized to the amplitude of scalar perturbations inferred by CMB measurements

(As ≈ 2.2 × 10−9). The local transformation is performed before the inclusion of the matter transfer

function, and the initial particle positions and velocities are finally computed from Φ(x) according to

the Zel’dovich approximation, as in the original Gaussian initial condition generator.

This code was tested by comparing simulations and theoretical predictions for the effect of local

primordial non-Gaussianity on the halo mass function and matter power spectrum (Desjacques, Seljak

& Iliev 2009). It has also been used to quantify the impact of local non-Gaussian initial conditions on

the halo power spectrum (Desjacques et al., 2009; Desjacques & Seljak, 2010) and bispectrum (Sefusatti

et al., 2010), as well as the matter bispectrum (Sefusatti et al., 2011). Fig. 2.13 shows the late time

power spectrum of two RANGER4000 realizations that started off the same initial power spectrum, but

one of which had non-Gaussian features set to fNL = 50. We see that the difference between the two

power spectra is at the sub-per cent level, and that the ratio of the two power spectra is well described

with one loop perturbation theory (Scoccimarro et al., 2004; Taruya et al., 2008).

2.8.2 Particle identification tags

A system of particle identification can be turned on, which basically allows to track each particle’s

trajectory between checkpoints. Such a tool is useful for a number of applications, from reconstruction

7We remind that for Zel’dovich approximation to hold in such cases, the simulations need to be started at very earlyredshifts. Consequently, the size of the first few redshift jumps in such simulations can become rather large, and thereforeless accurate, which is why we must carefully constrain the size of the jumps, as discussed in section 2.6.3. Another solutionto this issue is to use second order perturbation theory to generate the initial conditions, which is not implemented yet inthe public package.


Figure 2.13: Dark matter power spectrum, measured at z = 0 in a volume 3.2h−1Gpc per side, from 40003 particles.

The two curves represent two RANGER4000 realizations of the same initial power spectrum, one of which used Gaussian

statistics (blue in the online version) and the other the non-Gaussian initial condition generator (red online). The two

curves differ at the sub-per cent level, as seen in the top panel, and the one-loop perturbation theory calculations (aqua

online) accurately describes the ratio between the two curves up to k ∼ 0.4hMpc−1 in this case.


of halo merging history to tracking individual particle trajectories. The particle tag system has been

implemented as an array of double precision integers, PID, and assigns a unique integer to each particle

during the initialization stage. Since the array takes a significant amount of memory, we opted to store

the tags in a separate object – as opposed to adding an extra dimension to the existing xv array, such

that it can be turned off when memory becomes an issue. Also, it simplifies many post-processing

applications that read the position-velocity only. The location of each tag on the PID array matches

the location of the corresponding particle on the xv array, hence in practice it acts just like an extra

dimension. The tags on the array change only when particles exit the local volume, in which case the tag

is sent along the particle to adjacent nodes in the pass particle subroutine; similarly, deleted particles

result in deleted tags. As for the xv arrays, the PID arrays get written to file in parallel at each particle

checkpoint.

2.8.3 Extended range of the pp force

As discussed in section 2.6.2, one of the main sources of error in the calculation of the force occurs from

the PM interaction at the smallest scales of the fine grid. The NGP approximation is the least accurate

there, which causes a maximal scatter about the exact 1/r2 law. A straightforward solution to minimize

this error consists in extending the pp force calculation outside a single cell, up to a range where the

scatter is smaller. Although this inevitably reintroduces a number of operations that scales as N2, our

goal is to add the flexibility to have a code that runs slower, but produces results with a higher precision.

To allow this feature, we have to choose how far outside a cell we want the exact pp force to be active.

Since the force kernels on both meshes are organized in terms of grids, the simplest way to implement

this feature is to shut down the mesh kernels in a region of specified size, and allow the pp force to

extend therein. Concretely, these regions are constructed as cubic layers of fine mesh grids around a

central cell; the freedom we have is to choose the number of such layers.

To speed up the access to all particles within the domain of computation, we construct a thread safe

linked list to be constructed and accessed in parallel by each core of the system, but this time with a

head-of-chain that points to the first particle in the current fine mesh cell. We then loop over all fine

grids, accessing the particles contained therein and inside each fine grid cell for which we killed the

mesh kernels, we compute the separation and the force between each pair and update their velocities

simultaneously with Newton’s third law. To avoid double counting, we loop only over the fine mesh

neighbours that produce non-redundant contributions. Namely, for a central cell located at (x1, y1, z1),

we only consider the neighbours (x2, y2, z2) that satisfy one of the following conditions:


• z2 ≥ z1

• z2 = z1 and y2 ≥ y1

• z2 = z1 and y2 = y1 and x2 > x1

The case where all three coordinates are equal is already calculated in the standard pp calculation of

the code, hence we don’t recount it here. To assess the improvement of the force calculation, we present

in Fig 2.14 a force versus distance plot analogous to Fig. 2.7, but this time the pp force has been

extended to two layers of fine cells (again in a CITA128 realization). We observe that the scatter about

the theoretical curve has reduced significantly, down to the few percent level, and is still well balanced

around the theoretical predictions. The fractional error on the radial and tangential components of the

force, as seen in the right panel, are now at least five times smaller than in the default P3M algorithm.

When averaging over 10 time steps, we observe that improvement is relatively mild, showing that the

calculations are already very accurate.

To quantify the accuracy improvement versus computing time requirements, we perform the following

test. We generate a set of initial conditions at a starting redshift of z = 50, with a box size equal to

150h−1Mpc, and with 5123 particles. We evolve these SciNet512 realizations with different ranges for

the pp calculation, and compare the resulting power spectra. For the results to be meaningful, we again

need to use the same seeds for the random number generator, such that the only difference between

different runs is the range of the pp force. Fig. 2.15 shows the dimensionless power spectrum of the

different runs. We first see a significant gain in resolution when extending PM to P3M; gains roughly

as strong are found when adding successively one and two layers of fine cell mesh in which the pp force

is extended. We have not plotted the results for higher numbers of layers, as the improvement becomes

milder there while the runs take increasingly more time to complete: For this reason, it seems that a

range of two layers suffices to reduce most of the undesired NGP scatter.

Extending the pp calculation comes at a price, since the number of operations scales as N2 in the

sub-domain. This cost is best captured by the increase of real time required by a fixed number of

dedicated CPUs to evolved the particles to the final redshift. For instance, in our SciNet512 simulations,

the difference between the default configuration and Nlayer = 1 is about a factor of 2.78 in real time,

and about 6.5 for Nlayer = 2. This number will change depending on the problem at hand and on the

machine, and we recommend to perform performance gain vs resource usage tests on smaller runs before

running large scale simulations with the extended pp force.

The power spectrum does not provide the complete story, and one of the most relevant ways to

quantify the improvement of the calculations is to compare the halo mass function from these different


100

101

102

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

r[cells]∆

F/F

radialtangential

100

101

102

−0.35

−0.3

−0.25

−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

r[cells]

∆F

/F

radialtangential

Figure 2.14: (top left:) Gravity force in the P3M algorithm, compared with the exact 1/r2 law, in the same CITA128

realization as that shown in Fig. 2.7, except that the exact pp force has been extended to two fine mesh layers around each

particle in the force test code. Particles in that range follow the exact curve, after which we observe a scatter at distances

of the order of 2 fine grid that is much smaller than that observed in Fig. 2.7. The NGP interpolation scheme is again

responsible for the scatter, but the effect is suppressed for increasing distances. (top right:) Fractional error on the force

in the radial direction (points), over plotted with the scatter in the fractional tangential contribution (solid line). This was

also obtained over a single time step. (bottom row:) Same as top row, but averaged over ten time steps.


10−1

100

101

10−1

100

101

102

k[h/Mpc]

∆2(k

)

PM onlyp3mN layers = 1N layers = 2

101

101.3

101.5

101.7

101.9

Figure 2.15: Dimensionless power spectrum for varying ranges of the exact pp force. These SciNet512 realizations

evolved from a unique set of initial conditions at a starting redshift of z = 100 until z = 1.0, with a box size equal to

150h−1Mpc. The only difference between the runs is the ranges of the pp calculation. The inset shows details about the

resolution turnaround, and the thick vertical line corresponds to the coarse mesh scale.

SciNet512 runs. Fig. 2.16 presents this comparison at redshift z = 1.0. About 76,000 haloes were

found in the end, yielding a halo number density of about 0.0225 per [Mpc/h]3. We observe that the

simulation undershoot the Sheth-Tormen predictions, which is caused by the relatively low resolution of

the configuration compared to the RANGER4000 run. We also observe that the pure PM code yields up

to 10 per cent less haloes than the P3M version over most of the mass range, whereas the extended pp

algorithm generally recovers up to 10 per cent more haloes in the range 1011 − 1013M⊙. The difference

in performance between an extended pp range of two vs four cells deep is rather mild, from where we

conclude that Nlayers = 2 is the optimal choice.

2.9 Conclusion

This paper describes CUBEP3M, a public and massively parallel P3M N-body code that inherits from

PMFAST and that now scales well to 20,000 cores, pushing the limits of the cosmological problem size

one can handle. This code is fast and has a memory footprint up to three times smaller than Lean-

GADGET-2. We summarize the code structure, review the double-mesh Poisson solver algorithm, and

present scaling and systematic tests that have been performed. We also describe various utilities and

extensions that come with the public release, including a run time halo finder, an extended pp force

calculation, a system of particle ID and a non-Gaussian initial condition generator. CUBEP3M is one


1011

1012

1013

1014

10−3

10−2

(M2/ρ)d

n/dM

STPM onlyp3mN layers=1N layers=2

1011

1012

1013

1014

0.6

0.8

1

1.2

Rati

o

M [M⊙]

Figure 2.16: (top:) Halo mass function for different ranges of the pp force calculation, compared to the predictions of

Sheth-Tormen, at z = 1.0. The smallest haloes shown here have a mass equivalent to 20 particles of 2.79 × 109M⊙ each,

and fall in the lowest mass bin. (bottom:) Ratio between the different curves and that of the default P3M configuration.

of the most competitive N-body code that is publicly available for cosmologists and astrophysicists, it

has already been used for a large number of scientific applications, and it is our hope that the current

documentation will help the community in interpreting its outcome. The code is publicly available

on github.com under cubep3m, and extra documentation about the structure, compiling and running

strategy is can be found on the CITA wiki page8.

Acknowledgements

The CITA simulations were run on the Sunnyvale cluster at CITA. ITI was supported by the Southeast

Physics Network (SEPNet) and the Science and Technology Facilities Council grants ST/F002858/1 and

ST/I000976/1. Computations for the SciNet runs were performed on the GPC supercomputer at the

SciNet HPC Consortium. SciNet is funded by: the Canada Foundation for Innovation under the auspices

of Compute Canada; the Government of Ontario; Ontario Research Fund - Research Excellence; and

the University of Toronto. The authors acknowledge the TeraGrid and the Texas Advanced Computing

Center (TACC) at the University of Texas at Austin (URL: http://www.tacc.utexas.edu) for providing

HPC and visualization resources that have contributed to the research results reported within this paper.

ITI also acknowledges the Partnership for Advanced Computing in Europe (PRACE) grant 2010PA0442

which supported the code scaling studies. ULP and JDE are supported by the NSERC of Canada, and

8wiki.cita.utoronto.ca/mediawiki/index.php/CubePM


VD acknowledges support by the Swiss National Science Foundation.

Chapter 3

Non-Gaussian Error Bars in Galaxy

Surveys-I

3.1 Summary

We propose a method to estimate non-Gaussian error bars on the matter power spectrum from galaxy

surveys in the presence of non-trivial survey selection functions. The estimators are often obtained from

formalisms like FKP and PKL, which rely on the assumption that the underlying field is Gaussian. The

Monte Carlo method is more accurate but involves the tedious process of running and cross-correlating

a large number of N-body simulations, in which the survey volume is embedded. From 200 N-body

simulations, we extract a non-linear covariance matrix as a function of two scales and of the angle

between two Fourier modes. All the non-Gaussian features of that matrix are then simply parameterized

in terms of a few fitting functions and Eigenvectors. We furthermore develop a fast and accurate strategy

that combines our parameterization with a general galaxy survey selection function, and incorporate

non-Gaussian Poisson uncertainty. We describe how to incorporate these two distinct non-Gaussian

contributions into a typical analysis pipeline, and apply our method with the selection function from the

2dFGRS. We find that the observed Fourier modes correlate at much larger scales than that predicted

by both FKP formalism or by pure N-body simulations in a ‘top hat’ selection function. In particular,

the observed Fourier modes are already 50 per cent correlated at k ∼ 0.1hMpc−1, and the non-Gaussian

fractional variance on the power spectrum (σ2P /P

2(k)) is about a factor of 3.0 larger than the FKP

prescription. At k ∼ 0.4hMpc−1, the deviations are an order of magnitude.

45

Chapter 3. Non-Gaussian Error Bars in Galaxy Surveys-I 46

3.2 Introduction

With new galaxy surveys probing a larger dynamical range of our Universe, our ability to constrain

cosmological parameters is improving considerably. In particular, one of the most important goal of

modern cosmology is to understand the nature of dark energy (Albrecht et al., 2006), a challenging task

since there are currently no avenues for direct observations. It is however possible to probe its dynamics

via its equation of state ω, which enters in the Friedmann equation that governs the expansion of the

Universe. Among different ways ω can be measured, the detection of the baryonic acoustic oscillations

(BAO) dilation scale (Eisenstein et al., 2005; Tegmark et al., 2006; Hutsi, 2006; Percival et al., 2007;

Blake et al., 2011) is one of the favourite, both because of the low systematic uncertainty and the

potentially high statistics one can achieve with current (Huchra et al., 1990; York et al., 2000; Colless

et al., 2003; Drinkwater et al., 2010) and future galaxy surveys (Peterson et al., 2006; Acquaviva et al.,

2008; Schlegel et al., 2009b; LSST Science Collaborations et al., 2009; Benıtez et al., 2009; Beaulieu

et al., 2010).

The strength of the BAO technique relies on an accurate and precise measurement of the matter

power spectrum, whose uncertainty propagates on to the dark energy parameters via a Fisher matrix

(Tegmark, 1997). It is thus of the utmost importance to have optimal estimators of both the mean and

the uncertainty of the power spectrum to start with. The prescription to construct an estimator for

the power spectrum of a Gaussian random field, in a given galaxy survey, was pioneered by Feldmann,

Kaiser and Peacock (Feldman et al., 1994) (FKP for short). It states that the survey selection function

effectively couples Fourier bands that are otherwise independent, and that the underlying power should

then be deconvolved (Sato et al., 2011b). This technique has been used in many power spectrum

measurement (Feldman et al., 1994; Percival et al., 2001; Cole et al., 2005; Hutsi, 2006; Blake et al.,

2010). Although it is fast, the error bars between the bands are correlated, plus it has the undesired

tendency to smear out the underlying power spectrum, which can effectively reduce the signal-to-noise

ratio in a BAO measurement. In that sense, the FKP power spectrum is said to be suboptimal.

The band correlation induced by the FKP prescription can be removed by an Eigenvector decom-

position of the selection function, following the Pseudo Karhunen-Loeve formalism (Vogeley & Szalay,

1996)(PKL). This was used in the analysis of the SDSS data (Tegmark et al., 2006) and is the most

optimal (i.e. loss-less) estimator for Gaussian random field, as understood from the information theory

point of view. It is nevertheless a well known fact that this Gaussian assumption about the field is only

valid in the linear regime, since the non-linear gravitational collapse of the density effectively couples

different Fourier modes together (Meiksin & White, 1999; Rimes & Hamilton, 2005), and the phases


of the modes are no longer random (Coles & Chiang, 2000). Both the FKP and PKL prescriptions,

by their Gaussian treatment, do not take into account the intrinsic non-linear coupling of the Fourier

modes. It follows from this that for both methods, the measured power spectrum is suboptimal and

the error bars are systematically biased. Although the bias is usually small, it causes a problem when

estimating derived quantities that need to be measured with per cent level accuracy.

For instance, the observed BAO signal sits right at the transition between the linear and the non-linear

regime, therefore an optimal estimator of the power spectrum must incorporate the non-linear modes.

In particular, constraints on dark energy from BAO measurements require an accurate measurement of

the matter power spectrum covariance matrix. Under the FKP and PKL formalisms, the covariance

matrix is biased as it tends to underestimate the uncertainty and the amount of correlation between

the power bands. Alternative ways of estimating the error, i.e. methods that involve mock catalogues,

do model these non-linear dynamics, but it is not clear that the results are precise enough to measure

four-points statistics, and we rather rely on accurate N-body simulations.

Even more relevant is the recent realization that an optimal, i.e. non-Gaussian, estimate of the BAO

dilation scale requires a precise measurement of the inverse of the matrix, which is challenging due the

noisy nature of the forward matrix. It was nevertheless shown that, by consistency, the error bars on

a suboptimal measurement of the power spectrum should be calculated in a manner that incorporates

some noise in the measurement of the mean (Ngan et al., 2012). Only an optimal measurement of the

mean power spectrum can be matched with the straightforward (i.e. noise-less) non-linear covariance

matrix, and it was shown in the same paper that both estimators differ by a few per cent.

When constructing an estimator of the covariance matrix that corresponds to the sensitivity of a

particular survey, the convolution with the survey selection function is one of the most challenging

part. Whereas the convolution of the underlying power spectrum can be operated with angle averaged

quantities, the convolution of the covariance matrix must be done in 6 dimensions, since the underlying

covariance is not isotropic: Fourier modes with smaller angular separations are more correlated than

those with larger angles (Chiang et al., 2002; Bernardeau et al., 2002). The first challenge is to measure

accurately this angular correlation, which is also scale dependent. Neither second order perturbation

theory nor log-normal densities have been shown to calculate this quantity accurately, we must therefore

rely on N-body simulations. This requires a special approach, since a naive pair counting of all Fourier

modes in the four-point function, at a given angle, would take forever to compute. The second challenge

comes from the 6-dimensional convolution of the covariance matrix with the survey function. This is a

task that current computer clusters cannot solve by brute force, so we must find a way to use symmetries

of the system and reduce the dimensionality of the integral. The fact is that the underlying covariance


really depends only on three variables: two scales and the relative angles between the two Fourier modes.

Moreover, it turns out, as we describe in section 3.7, that it is possible to express this matrix into a set

of multipoles, each of which can further be decomposed into a product of Eigenvectors. This effectively

factorizes the three dimensions of the covariance, hence the convolution can be broken down into smaller

pieces. By doing so, the non-Gaussian calculation is within reach, and we present in this paper the first

attempt at measuring deviations from Gaussian calculations, including both Poisson noise and a survey

selection function. In short, the main ideas of this paper can be condensed as follow:

1. The underlying non-linear covariance matrix of the matter power spectrum exhibits many non-

Gaussian features in the trans- and non-linear regimes. First, the diagonal elements of the angle-

averaged covariance grow stronger, and correlation across different scales becomes important. Sec-

ond, Fourier modes with similar (or identical) magnitudes correlate more if the angle between them

is small.

2. It is possible to model all of the aboved mentioned non-Gaussian aspects with a small number of

simple functions.

3. With such a parameterization, it is possible, for the first time, to solve the six-dimensional integral

that enters the convolution of the covariance of the power spectrum with the galaxy survey selection

function.

Concerning the second point, the parameters that best fit our measurements are provided in section 3.8,

but these are separately testable, and could be verified by other groups and in other ways. These are

anyway expected to change when one uses haloes instead of particles. The third point is, however, a

straightforward recipe that is robust under possible changes of best-fitting parameters, and provides, as-

suming that the input parameters are correct, an unbiased measurement of the non-Gaussian uncertainty

of the matter power spectrum.

Our first objective is thus to measure the covariance of the power spectrum between various scales

and angles, and organize this information into a compact matrix, C(k, k′, θ). We describe how we solve

this problem in a fast way, which is based on a series of fast Fourier transforms that can be run in

parallel on a large number of computers. We find that the angular dependence, at fixed scales (k 6= k′),

is rather smooth, it agrees with analytical predictions in the linear regime, but deviates importantly from

Gaussianity for smaller scales. The dependence is somehow similar when the two scales are identical, up

to a delta function for vanishing angles. We also found that, once projected on to a series of Legendre

polynomials, it takes very few multipoles to describe the complete original function. We perform this

transform for all scale combinations and group the results in terms of multipole moments.


Our second objective is to provide a general method to combine this C(k, k′, θ) with a survey selection

function and non-Gaussian Poisson noise, and hence allow the extraction of non-Gaussian error bars

on the measured power spectrum. We test our technique on the publicly available 2dFGRS selection

functions (Norberg et al., 2002) and find that there is a significant departure between the Gaussian

and non-Gaussian treatment. In particular, the fractional error of the power spectrum (σ2P /P

2(k))

at k ∼ 0.1hMpc−1 is about a factor of 3.0 higher in the non-Gaussian analysis, and the departure

reaches an order of magnitude by k ∼ 0.4hMpc−1. The method proposed here can be also applied to

other kinds of BAO experiments, including intensity mapping from the emission of the 21 cm line by

neutral Hydrogen (Peterson et al., 2006; Lazio, 2008; Schlegel et al., 2009b), or Lyman-α forests surveys

(McDonald & Eisenstein, 2007; McQuinn & White, 2011). We did not, however, include the effect

of redshift distortions, and focused our efforts on dark matter density fields obtained from simulated

particles. An improved version of this work would include both of these effects, however.

As indicated by the title, this paper is the first part of a general strategy that aims at constructing

unbiased, non-Gaussian estimators of the uncertainty on the matter power spectrum measured in galaxy

surveys. The second part, which we hereafter refer to as HDP2 (in preparation), exploits the fact that the

measurement of the C(k, k′, θ) matrix provides a novel handle at measuring C(k, k′): the two quantities

are related by a straightforward integration over θ. As shown in a later section of the current paper, it

turns out that the main contributions to C(k, k′) come from small angles, while larger angles are noise

dominated. It is thus possible to perform a noise weighted integral, which results in a more optimal

measurement of C(k, k′) and of its error bars, compared to direct or bootstrap sampling. We can then

extract accurate non-Gaussian error bars on the power spectrum with fewer realizations, which opens

the door for an error estimate directly from the data (i.e. an internal estimate), a significant step forward

in the error analysis of galaxy surveys.

The current paper is organized as follow: in section 3.3, we briefly review the FKP method, and

describe how to estimate non-Gaussian error bars in realistic surveys, given a previous knowledge of

C(k, k′, θ). We then lay down the mathematical formalism that describes how we extract this quantity

from simulated density fields in section 3.4. Section 3.5 describes sanity checks, null tests, and our N-

body simulations. We present our measurements of C(k, k′, θ) in section 3.6, and describe the multipole

decomposition in section 3.7. In section 3.8, we further simplify the results by extracting the principal

Eigenvectors and provide fitting formulas to reconstruct easily the full covariance matrix. Section 3.9

contains results of applying our method for a set of simple selection functions. We finally discuss some

implications and extensions of our methods in section 3.10, and conclude in section 3.11.


3.3 Matter Power Spectrum from Galaxy Surveys

In this section, we quickly review the general FKP method, which is commonly used in data analysis

(Feldman et al., 1994; Percival et al., 2001; Blake et al., 2010). We then point out some of the major flaws

of such techniques when measuring the uncertainty, and describe how non-Gaussian error bars could be

estimated in principle. Before moving on, though, we first lay down the conventions used throughout

the paper. The reader familiar with the FKP method may skip to section 3.3.2.

A continuous density field δ(x) is related to its Fourier transform δ(k) by

δ(k) =

∫δ(x)eik·xd3x (3.1)

where k is the wave number corresponding to a given Fourier mode. The power spectrum P (k) of the

field is defined as:

〈δ(k)δ∗(k′)〉 = (2π)3P (k)δD(k− k′) (3.2)

and is related to the mass auto-correlation function by :

ξ(x) =1

(2π)3

∫e−ik·xP (k)d3k (3.3)

In the above expressions, the angle brackets refer to a volume average in Fourier space, and δD(k) stands

for the Dirac delta function.

3.3.1 The optimal estimator of the power spectrum

The power spectrum of the matter field contains a wealth of information about the cosmic history and the

principal constituents of the Universe. Unfortunately, it is not directly detectable, since our observations

are subject to cosmic variance, detection noise, light to mass bias, redshift distortions and incomplete

sky surveys. The FKP method provides an optimal estimator of the matter power spectrum P (k) under

the assumption that the density field is Gaussian. It is formulated in terms of the survey selection

function W (x), the galaxy number density n, the dimensions (nx, ny, nz) of the grid where the Fourier

transforms are performed, and the actual number count per pixel n(x). All the following calculations

can be found in Feldman et al. (1994), and are included here for the sake of completeness.

The first step is to construct series of weights w(x) as

w(x) =1

1 +W (x)NcnP0=

1

1 + nP0(3.4)

where Nc = nxnynz , n is the mean galaxy density and P0 is a characteristic amplitude of the power

spectrum at the scale we want to measure. Since the latter is not known a priori, it is usually obtained


from a theoretical model, and sometimes updated iteratively. The selection function is also normalized

such that∑

xW (x) = 1.

The optimal estimator of the power spectrum, Pest(k), is obtained first by re-weighting each pixel

by the weights in [Eq. 3.4], then by subtracting from the result a random catalogue with the same

selection function, weights and number of objects N . After taking the expectation value of the results,

the 2-points statistics of the pixel counts becomes

〈n(x)n(x′)〉 = nn′(1 + ξ(x− x′)) + nδD(x− x′) (3.5)

where n is the mean density in the patch over which the average is performed. The Fourier transform is

then given by

〈Pest(k)〉 =|n(k)−NW (k)|2 −N

∑xW (x)w2(x)

N2Nc

∑xW

2(x)w2(x)(3.6)

where the denominator is a convenient normalization. This measured power is aliased by the grid mass

assignment scheme, and should be divided by the appropriate function (Jing, 2005).

What this estimator measures is not the underlying power spectrum P (k), but a convolution with

the survey selection function:

〈Pest(k)〉 =∑

k′ P (k′)|W (k − k′)|2Nc

∑xW

2(x)w2(x)(3.7)

It ideally needs to be deconvolved, an operation that is not always possible.

For many survey geometries, the convolution effectively transfer power across different bins which

are uncoupled to start with (Tegmark et al., 2006). As mentioned previously, the PKL prescription also

assumes that the density field is Gaussian, but rotates into a basis in which the bins are decoupled.

In that sense, the PKL technique is more optimal than the FKP, unless the selection function is close

to a “top hat”, in which case the induced mode coupling vanishes. Both cases, however, rely on the

fundamental assumption that the underlying density field is Gaussian, which is known to be inaccurate

in the trans- and non-linear regime, where one still wants an accurate measure of the power spectrum

for a BAO analysis. Obtaining accurate error bars is a requirement for optimal analyses, and we shall

examine how these are usually obtained.

3.3.2 The FKP covariance matrix

The covariance matrix of the angle averaged power spectrum is a four point function that contains the

information about the band error bars, and possible correlation between them. As mentioned earlier, it

is required for many cosmological parameter studies. It is generally obtained from the power spectrum


as

C(k, k′) = 〈∆P (k)∆P (k′)〉 (3.8)

where ∆P (k) refers to the fluctuations of the measured values about the mean, which is ideally obtained

from averaging over many realizations. In a typical galaxy survey, such independent realizations are

obtained by sampling well separated patches of the sky. Because of the cost of such an operation, the

number of patches is usually very small. The covariance matrix is thus not resolved from the data,

and the error bars are obtained with external techniques, i.e. from mock catalogues1, or directly from

Gaussian statistics (see HDP2 for a prescription that overcomes this challenge). For a uniform (top-hat)

selection function, the Gaussian covariance matrix is estimated as:

CGauss(k, k′) =2

N(k)(P (k) + Pshot)

2δkk′ (3.9)

where Pshot = 1/n and N(k) is the number of Fourier modes that enters in the measurement of P (k). In

the ideal scenario of perfect spherical symmetry and resolution, N(k) = 4πk2∆k(

L2π

)3, with ∆k being

the width of the k-band. The Kronecker delta function ensures that there is no correlation between

different modes, an inherent property of Gaussian random fields. This equation can easily be modified

to deal with measurements without angle averaging.

The FKP prescription provides a generalization of [Eq. 3.9] for the case where the selection function

varies across the volume. It is obtained from [Eq. 3.6] and given by

CFKP (k,k′) =2

N(k)N(k′)

∑

k,k′

|PQ(k− k′) + S(k− k′)|2 (3.10)

where

Q(k) =

∑xW

2(x)w2(x)exp(ikx)∑xW

2(x)w2(x)(3.11)

S(k) =

(1

nNc

) ∑xW (x)w2(x)exp(ikx)∑

xW2(x)w2(x)

(3.12)

In [Eq. 3.10], P is taken to be the mean of the power spectrum at separation k−k′. Because the selection

functions are usually quite compact about k = 0, that approximation is reasonable for Gaussian fields.

Also, [Eq.3.9] can be recovered by setting W (x) = 1/Nc.

3.3.3 Non-Gaussian error bars

As mentioned in the last section, it is necessary to have access to many realization of the matter field in

order to measure a non-Gaussian covariance matrix of power spectrum. This could in principle be done

1We post-pone the discussion on mock catalogues until the next section


from data across many different patches in the sky, but even then, we have only one sky to resolve the

largest modes, which would therefore be dominated by cosmic variance. Not to mention the cost and time

involved in measuring many large but disconnected volumes. Fortunately, N-body simulations are now

accurate and fast enough to generate large numbers of measurements of the matter power spectrum.

Since they model the non-linear dynamics of structure growth, the density fields they generate are

non-Gaussian. The covariance matrix constructed from a high number of simulations indeed shows a

correlation across different scales in the non-linear regime (Meiksin & White, 1999; Rimes & Hamilton,

2005; Takahashi et al., 2009; Ngan et al., 2012).

Although much more representative of the underlying covariance, such matrices are hard to incor-

porate in a data analysis, first because they are based on a fixed set of cosmological parameters, but

also because the simulated volume is cubic and periodic. Each survey group typically needs to run at

least one N-Body simulation, and measure the power spectrum with and without the measured selec-

tion function, in order to quantify the bias of their measurement. The complete approach would then

be to run hundreds of these to measure the covariance matrix, and that over a range of cosmological

parameters values. This whole procedure is expensive, which explains why it is never done in prac-

tice. The alternative is to use mock galaxy catalogues, obtained, for example, from log normalization

of Gaussian densities, second order perturbation theory (PT), haloPT, and so on. Unfortunately, the

accuracy of such techniques at modelling the four-point functions and angle dependencies has not been

fully quantified.

Another artefact of the simulations is that the number of particles can be arbitrarily adjusted such

as to suppress the Poisson noise down to a level where it is negligible. This is certainly not true for many

galaxy survey, in which the number density is often much lower. We measure a non-Gaussian Poisson

error by sampling random fields with a selection threshold chosen as to mimic the number density of a

realistic survey, and incorporate the effect manually in the analysis, as explained in section 3.9.

To measure non-Gaussian error bars on a realistic survey, the most accurate procedure would be to

convolve the best available estimator of the covariance matrix with the selection function. Because the

latter is generally not spherically symmetric, it is the full 6-dimensional covariance matrix, C(k,k′), that

needs to be integrated over. Let us suppose, for a moment, that we successfully measured that complete

non-Gaussian covariance matrix. It would first contain an element for each Fourier modes k (i.e. with

no angular averaging), and from [Eq. 3.7 and 3.8], we can write:

Cest(k,k′) =

∑k′′,k′′′〈∆P (k′′)∆P (k′′′)〉|W (k − k′′)|2|W (k′ − k′′′)|2

(N2Nc

∑xW

2(x)w2(x))2(3.13)

where the angled bracket is nothing else but that full covariance matrix C(k′′,k′′′). We can then simplify


the result since the covariance between two Fourier modes depends only on the angle γ between them,

and not on the absolute orientation of the pair in space. In other words, we make use of this symmetry

argument to write C(k′′,k′′′) = C(k′′, k′′′, γ) without lost of generality. This angle can further be

expressed in terms of the two angles made by k′′ and k′′′ as

cosγ = cosθ′′cosθ′′′ + sinθ′′sinθ′′′cos(φ′′ − φ′′′) (3.14)

We show in a later section of this paper that the true covariance matrix can be decomposed into

a sum of factorized terms, each of the form F1(k′′)F2(k

′′′)G1(θ′′, φ′′)G2(θ

′′′, φ′′′). Therefore the double

convolution of [Eq. 3.13] can actually be broken into a sum of smaller pieces, with at most 3-dimensional

integrals to perform.

3.4 Measuring the Angular Dependence: the Method

As mentioned above, our first objective is to extract the covariance matrix of the power spectra from

N-Body simulations, as a function of two scales and one angle: C(k, k′, θ). In this section, we develop a

novel way to obtain covariances and cross-correlations and which allows us to perform this measurement.

3.4.1 Cross-correlations from Fourier transforms

We begin by assuming we have measured the power spectrum from a large number of simulations. We

first compute the mean of the angle averages: P (k) ≡ 〈P (k)〉N,Ω and the deviation from the mean of

each mode:

∆P (k) = P (k)− P (k) (3.15)

We then select two scales, ki and kj , that we want to cross-correlate. We make two identical copies of

three-dimensional power spectra and multiply each one by a radial top hat function corresponding to

the particular scales:

∆Pi(k) ≡ ∆P (k)ui(|k|) (3.16)

where ui(k) = θ(k − ki)θ(−k + ki + δk) is the product of two Heaviside functions. Also, δk is the shell

thickness, taken to be very small. We then cross-correlate the subsets and define:

Σij(∆k) =1

(2π)3

∫∆Pi(k)∆Pj(k+∆k)d3k (3.17)

We then express both ∆Pi,j(k)’s in [Eq. 3.17] in terms of their mass auto-correlation functions ∆ξi,j(x).

We first integrate over exp[ik · (x+x′)]d3k and obtain a delta function, which allows us to get rid of one


of the real space integral. After slightly rearranging the terms, we obtain:

Σij(∆k) =

∫∆ξi(x)∆ξ

∗j (x)e

−i∆k·xd3x (3.18)

In the above equation, ∆ξi can be expressed as:

∆ξi(x) =1

(2π)3

∫e−ik·x∆P (k)ui(|k|)d3k

=1

(2π)3

∫ ki+δk

ki

k2dk

∫e−ik·x∆P (k)dΩ (3.19)

Since the shells we select are very thin, we can safely approximate that the power spectrum is constant

over the infinitesimal range, and thus perform the k integral:

∆ξi(x) =1

(2π)3k2i δk

∫e−iki·x∆Pi(k)dΩ (3.20)

We repeat the same procedure for the scale j, multiply both auto-correlation functions together, and

Fourier transform the product, following [Eq. 3.18]. The result is the cross-correlation Σij(∆k), which

becomes, after performing the x integral over the plane wave:

Σij(∆k) =1

(2π)3k2i k

2j δ

2k

∫dΩ

∫dΩ′ ×∆Pi(k)∆Pj(k

′)δD(k′j − ki −∆k) (3.21)

The delta function enforces ∆k to point from ki to k′j. This geometry allows us to use the cosine law

and relate |∆k| to the angle θ it subtends, as seen in Fig. 3.1, such that:

θ = cos−1

(k2j + k2i − |∆k|2

2kjki

)(3.22)

Since many ∆k subtend the same angle θ, we can perform an average over them and compute

Σij(θ) ≡ 〈Σij(∆k)〉∆k=∆k (3.23)

3.4.2 Normalization

The quantity Σij(θ) is not exactly equal to C(ki, kj , θ), because there is a subtle double counting effect

which is purely geometrical, and which needs to be cancelled. To see how this arises, we work out

a very simple scenario, in which the density field is perfectly isotropic. In that case, we can write

∆P (k) = ∆P (k), hence the angular integration in [Eq.3.20] is straightforward and we get:

∆ξi(x) = ∆ξi(x) =k2iπL

∆Pi(k)j0(kix) (3.24)

with j0(x) being the zeroth order spherical Bessel function. We have also assigned δk = 2π/L to the

shell thickness, which corresponds to the resolution of a simulation of side L. Then, [Eq.3.18] becomes

Σij(θ) =

(kikjπL

)2

∆P (ki)∆P (kj)Fij(θ) (3.25)


where

F ij(θ) =

∫j0(kix)j0(kjx)j0(θx)x

2dx (3.26)

The function F (ki, kj , θ) is independent of the actual power spectrum; it is purely a geometrical artefact

that corresponds to the counting of the different combinations of ki,j that produce a given ∆k. As the

former increase, so does the surface of the k-shells, hence there are more ways to fit ∆k. In the case of

an exactly isotropic power spectrum, the results should have no angular dependence. We thus define a

normalization ΣijN (θ), as the output of [Eq. 3.23] with ∆P (ki,j) = 1 everywhere on the shells. The final

results are obtained by dividing off this normalization, which cancels off the geometrical effect:

C(ki, kj , θ) ≡Σij(θ)

ΣijN (θ)

= 〈∆P (ki)∆P (kj)〉 (3.27)

We stress again that this result is an average over all configurations satisfying kj = ki +∆k.

To summarize, here is a condensed list of the steps taken to measure C(k, k′, θ):

1. Measure the mean angle averaged P (k) from an ensemble of simulations,

2. Select a combination of shells ki,j to cross-correlate,

3. For each simulation, compute P (k), duplicate and multiply each replica by a top hat ui,j(k), which

effectively sets to zero every off-shell grid cells,

4. Subtract P (k) from each cell in the shell,

5. Fourier transform both grids, complex multiply them, and Fourier transform back to k-space,

6. Loop over the ∆k available, bin into Σ(|∆k|2), and express the results as a function of θ,

7. Repeat steps (v-vi), but this time assigning the value of each cell in the shell to unity, and divide

Σ(θ) by this normalization. This is a measure of C(ki, kj , θ) from one simulation,

8. Repeat for all simulations, then compute the mean,

9. Iterate over steps (ii-viii) for other shell combinations.

To achieve better results, we make use of the fact that P (−k) = P (k), hence, following [Eq.3.17],

we can write Σij(−∆k) = Σij(∆k). This translates into a theoretical symmetry about θ = π/2 in

the angular dependence of the covariance. That property turns out to be very useful for reducing the

numerical noise, since we can measure the covariance over the full angular range, but fold the results on

to 0 < θ < π/2. Also, to avoid interpolating error, we choose to bin in (∆k)2 before transforming to θ.


θ

~ki

~kj~∆k

Figure 3.1: Geometry of the system. For a fixed pair of shells, the magnitudes of the Fourier modes ki and kj are fixed,

so the angle between them is directly found from the separation vector ∆k. Note that we use interchangeably numbers or

roman letters to denote individual Fourier modes.

3.4.3 Zero-lag point

It is important to note that for a given realization, the point at θ = 0, which we refer to as the zero-lag

point, must be treated with care. When the two shells are identical, i.e. i = j, the zero-lag point of

each simulation first computes the square of the deviation the mean P (k), then averages the result over

the whole shell. It is equivalent to calculating the variance over the shell, but using a mean which is

somewhat off from the actual mean on that shell. That effectively boosts the variance. When we average

over all simulations, the zero-lag points can be written as:

Σii(0) = 〈P 2i (k)〉N,Ω − 〈P (ki)〉2N,Ω (3.28)

where, in the first term, the angle average and mean over all realizations are computed after squaring

each grid cell. By comparison, the variance on angle averaged power spectra would be obtained by

performing, in the first term, the angle averaging first, then taking the square, then taking the mean.

When the two shells are different, the zero-lag point is now the average over ∆P (k)∆P (k′) on both

shells. Since we are no longer squaring each terms, it now includes negative values, hence is generally of

much smaller amplitude.


3.5 Validation of the Method

We describe in this section a series of tests that compare our numerical results to semi-analytical solu-

tions. We apply our method on a few simple situations in which we control either the density field or the

three-dimensional power spectrum. We first test our recipe on a power spectrum that is set to the second

Legendre polynomial. The outcome can be compared to semi-analytical calculations and gives a good

grip on the precision we can achieve. We next measure the angular dependence of the covariance matrix

of white noise densities and present an estimate of the non-Gaussian Poisson error2. We finally measure

the angular cross-correlation from Gaussian random fields in order to better understand departures from

Gaussianity.

3.5.1 Testing C(k, k′, θ) with a Legendre polynomial

As a first test, we enforce the z-dependence of the power spectrum to be equal to the second Legendre

polynomial, and then compare our results to semi-analytic predictions. We manually set P (k) = k2z ,

which is thus constant across the x− y plane. The mean and the deviation from the mean on a shell ki

are given by 〈P (k)〉Ω = k2/3 and ∆P (k) = (2/3)k2P2(µ) respectively, where Pℓ(x) is the ℓ-th Legendre

polynomial and µ is the cosine of the inclination angle. The mass auto-correlation function associated

with this power is

∆ξi(x) =−2k4i6πL

j2(kix) (3.29)

The angular dependence of the covariance can be calculated semi-analytically from [Eq. 3.18] and [Eq.

3.29]. The angular integration is straightforward, and we obtain

Σij(∆k) =4k4i k

4j

9πL

∫ ∞

0

j2(kix)j2(kjx)j0(∆kx)x2dx (3.30)

We perform the x integral with ki=j = 1.0hMpc−1, repeat the procedure for ΣijN (∆k), and obtain

a semi-analytical prediction: C(k, k′, θ) ∼ P2(cosθ), up to numerical noise. This agrees well with the

numerical results produced by our technique, as shown in the top part of Fig. 3.2. We are plotting the

angle dependence of the covariance matrix, normalized by the angle average of the covariance, such that

the curve represents the actual cross-correlation coefficient between the Fourier modes. We mention here

that in the case where ki 6= kj , which we encounter in the following sections, we normalize to the square

root of the product of the corresponding matrix elements:

r(ki, kj , θ) =C(ki, kj , θ)√

C(ki, ki)C(kj , kj)(3.31)

2See Cohn (2006) for a discussion on different types of noise in a cosmological context.


−0.5

0

0.5

1

Legendre

−0.4

−0.2

0

0.2

0.4

r(k

i,k

j,θ

)

Poisson

10 20 30 40 50 60 70 80 90

0.04

0.06

0.08

0.1

0.12

θ[deg.]

Gauss

Figure 3.2: (top:) Angular dependence of the covariance of a power spectrum set to the 2nd Legendre polynomial,

calculated here at ki=j = 1.0hMpc−1. The solid line is the semi-analytical prediction. The curve is normalized to the

value of the zero-lag point, such that it represents the actual cross-correlation coefficient between the Fourier modes. In

this case, modes that point in like-directions are strongly correlated. (middle :) Angular dependence of the power spectrum

cross-correlation coefficient measured from 200 Poisson sampled random fields. The error bars are obtained by 500 bootstrap

resampling. We have selected two k-shells i, j that are off by one grid cell: kj = ki + δk, with δk = 0.0314hMpc−1 and

ki ∼ 1.0hMpc−1. The distribution for i = j is similar in shape, except for the zero-lag point, which is much larger than

any other points, and the plateau that is slightly higher. The solid line in this figure is the predicted value, which is well

within the error bars. We have reproduced a similar plot for Poisson densities with 8.0 million peaks, which is also flat,

and find that the height of the plateau scales roughly as 1/n3, where n is the number of Poisson sampled objects. (bottom:)

Angular dependence of the power spectrum cross-correlation coefficient, measured from 200 Gaussian random fields, this

time with kj = ki + 5δk , and again ki ∼ 1.0hMpc−1. The theoretical prediction is zero, whereas we measure a constant

6 per cent correlation bias across all angles. We have verified that this bias is scale independent by changing ki,j .

In the particular case under study in this section, the Fourier modes separated by small angles are

strongly correlated by construction.

3.5.2 Testing C(k, k′, θ) with Poisson-sampled random fields

To measure the response of our code to white noise, we produce a set of 200 random density fields, each

with the same comoving volume of 200h−1Mpc. These are then Poisson sampled with a fixed sensitivity

threshold that is chosen such that ∼ 8000 peaks are counted on average. The standard deviation in the

measured P (k) decreases roughly as k−2, expected from the fact that the number of cells on a k-shell

grows as k2.

Because of the random nature of Poisson densities, the variance on a given shell should be roughly

constant across all directions. Moreover, after averaging over many realization, Poisson densities are in


principle statistically isotropic. We thus expect the measured angular dependence of the covariance to

be very close to flat, and, from [Eq.3.27], we estimate it should plateau at a value somewhat similar to

C(k, k′):

CPoisson(k, k′, µ) ∼ CPoisson(k, k

′) +Aδkk′δµ±1 (3.32)

where the two delta functions ensure that modes with different directions or scales do not couple together.

The constant A is much larger than CPoisson(k, k′), for reasons explained in section 3.4.3, but the precise

value is irrelevant to the current analysis. Fig. 3.3 shows the cross-correlation coefficient matrix for non-

Gaussian Poisson noise. We observe that the angle-averaged modes are correlated by more than 30 per

cent between scales smaller than k = 1.0hMpc−1. The reason for this feature is actually independent of

cosmology, even though the matrix has a look very similar to that measured from simulations3 . The

explanation lies in the fact that each of our Poisson densities do not have exactly the same number of

objects, hence the asymptotic value of P (k) is not a perfect match for all field. This slight scatter in

power translates into a correlation between the high k−modes of a given density field. This is in good

agreement with the predictions of Cohn (2006), which calculated that the Poisson sampling of Gaussian

fields induce non-Gaussian statistics, and that well separated scales can correlate significantly.

We then measure the angular dependence of the covariance for these 200 Poisson distributions, also

at k ∼ 1.0hMpc−1. We obtain a distribution that is indeed close to flat, and consistent with a uniform

10 per cent correlation, as shown in the middle plot of Fig. 3.2. As before, we have normalized the

plot such as to exhibit the angular cross-correlation. Because the zero-lag point is typically a few orders

of magnitude above the other points, we quote its value in the text or in the figures’ caption where

relevant, and resolve the structure of the other angles. The mean of the un-normalized distribution is

133.3Mpc6h−6, a 10 per cent agreement with our rough estimate. We have re-binned the distributions

on to a set of points that are optimal for the upcoming angular integration, as described in section 3.7.

3.5.3 Testing C(k, k′, θ) with Gaussian random fields

The next test consists in measuring the angular dependence of the covariance from of 200 Gaussian

random fields. We use 200 power spectra measured at z = 0.5, obtained from N-Body simulations

(section 3.6.1), to generate 200 fields. Similarly to the Poisson fields, we expect the distribution to be

overall flat, except for the zero-lag point. Because we choose not to Poisson sample these Gaussian

densities, the randomness should be such that near to perfect cancellation occurs between the different

3It is in fact arguable that such a matrix, constructed from a set of Poisson densities, could have better performancesat modelling the ‘true’ non-Gaussian covariance matrix, compared to the naive Gaussian approximation.


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Figure 3.3: Cross-correlation coefficient matrix, measured from the power spectra of 200 Poisson sampled random density

fields, selected to have 8000 peaks on average. The correlation in high k-modes is purely a counting artefact, as explained

in the text. This represents our estimate of the non-Gaussian Poisson uncertainty.

angles, and the plateau should be at zero. In the continuous case, the Gaussian covariance can be

expressed as

CGauss(ki, kj , µ) =2〈P (ki)〉2N(ki)

δijδµ,±1 (3.33)

where N(k) is the number of Fourier modes in the k-shell. For ki = kj , the zero-lag point contains

perfectly correlated power, so we expect it to have a very large value. As explained in section 3.4.3, we

cannot directly compare its value to 2P 2(k)/N(k), since the former is bin dependent, while the latter is

not. In the case where i 6= j however, the zero-lag point should drop down to numerical noise.

The measured angular dependence is presented in the bottom part of Fig. 3.2, where we see that

the distribution is flat and consistent with 6 per cent correlation. This indicates that our method suffers

from a small systematic bias and detects a small amount of correlation, in a angle independent manner.

We have repeated this measurement for different scales ki,j and obtained the same bias. We therefore

conclude that any signal which is smaller than this amount is bias dominated and not well resolved.

3.6 Measuring the Angular Dependence

In this section, we present our measurements of the angular dependence of the covariance in our 200

simulations. We explore different scale combinations and attempt to compare the outcome to expected

results whenever possible. In all figures, the error bars were obtained from 500 bootstrap resampling of


our simulations, unless otherwise specified.

3.6.1 N-body simulations

Since our Universe is not Gaussian at all scales relevant for BAO or weak lensing analyses, a robust error

analysis should be based on non-Gaussian statistics, and, as mentioned earlier, N-body simulations are

well suited to measure covariance matrices. Our numerical method is fast enough that, for fixed ki and

kj , we can compute the angular dependence of the covariance matrix in about one minute. The average

over 200 realizations can be done in parallel, hence producing all available combinations takes very little

time.

The simulations are produced by CUBEP3M (Merz et al., 2005), a public N-body code that is both

OPENMP and MPI parallel, which makes it among the fastest on the market4. We generate 200 Gaussian

distributions of 200 h−1Mpc per side, each with 2563 particles, starting at zi = 40, and evolve them until

z = 0.5. The simulations are run on the CITA Sunnyvale cluster, a Beowulf cluster of 200 Dell PE1950

compute nodes, each equipped with 2 quad cores Intel(R) Xeon(R) E5310 @ 1.60GHz processors. Each

node has access to 4GB of RAM and 2 gigE network interfaces. The power spectrum of these simulations

is shown in Fig. 3.4, and shows a good agreement with the non-linear predictions from CAMB (Lewis

et al., 2000), up to k ∼ 0.25hMpc−1. Beyond that scale, the structures are underestimated due to the

resolution limit of the simulations. For the rest of this paper, we only consider well resolved scales, in

occurrence those in the range k ∈ [0.314, 2.34]hMpc−1, which we organize into 75 linearly spaced bins.

3.6.2 Results

We present in Fig. 3.5 and 3.6 the angular dependence of the covariance between the power spectrum

of various scales. As explained in the previous section, the distributions are normalized such as to

represent the cross-correlation coefficient between modes separated by an angle θ. In the first figure,

both scales are selected to be identical, and vary progressively from k = 0.17hMpc−1 to 2.34hMpc−1.

Modes separated by an angle larger than 30o are less correlated at all scales, and the correlation is even

smaller for modes smaller than 0.5hMpc−1. These latter modes are grouped in larger bins due to the

higher discretization of the shells, and ideally one would like to run another set of simulation with larger

volumes to have a better resolution on those scales. However, these larger scales have very little impact

on the non-Gaussian analysis we are carrying, we therefore do not attempt to improve the situation. For

highly non-linear scales, the correlation between modes separated by angles smaller than 10o increases

4http://www.cita.utoronto.ca/mediawiki/index.php/CubePM


100

102

104

P(k

)[M

pc/

h]3

10−1

100

0

0.5

1

1.5

2

k[h/Mpc]

P(k

)/P

CA

MB

(k)

Figure 3.4: (top:) Power spectrum of 200 simulations, produced by CUBEP3M, compared to CAMB at z = 0.5

(solid line). The error bars are the 1σ standard deviation on the 200 measured P (k). We only include modes with

k ≤ 2.34hMpc−1 in this analysis, as indicated by the arrow in the figure. (bottom:) Ratio between the simulated and

predicted power spectra.

up to 55 per cent.

In the second figure, one of the two scale is held constant, at k = 0.61hMpc−1, while the other varies

over the same range. Modes separated by angles larger than 30o are less than 10 per cent correlated,

for all combinations of scales. When the two scales are of comparable size, the correlation climbs up to

values between 15 and 20 per cent for angles smaller than 15o.

This angular behaviour is enlightening, as it shows how the error between Fourier modes separated

only by a small angle tends to correlate first. Qualitatively, this validates the fact that in non-Gaussian

densities, quasi-parallel Fourier modes are probing essentially the same collapsed structures. When the

angle is closer to 90o, however, one mode could go along a filament and the other across it, producing

only weak correlations. It could thus be possible to construct a highly clustered density in which we

could observe an anti-correlation at 90o, provided we are not noise dominated.

This coherent behaviour is a clear sign that the non-linear structures underwent gravitational collapse,

and the departure from Gaussianity and white noise is obvious. Another signature of non-Gaussianity

is that even in the presence of a small offset between the scales, the small angle correlation has a value

higher than those at larger angles, because of the coupling between those scales. Fig. 3.6 shows this

effect.


10 20 30 40 50 60 70 80 900

0.1

0.2

0.3

0.4

0.5

r(k

i,k

j,θ

)

θ[deg.]

0.17 h/Mpc0.46 h/Mpc0.93 h/Mpc2.34 h/Mpc

Figure 3.5: Angular dependence of the power spectrum cross-correlation, measured from of 200 density fields, at ki=j =

0.17, 0.46, 0.93 and 2.34hMpc−1. The distribution exhibits a correlation of less than 10 per cent for angles larger than

about 30o. For scales smaller than 0.5hMpc−1, the correlation increases up to 15 per cent for angles smaller than 10o, and

to more than 40 per cent for smaller angle.

10 20 30 40 50 60 70 80 900.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

r(k

i,k

j,θ

)

θ[deg.]

0.14 h/Mpc0.46 h/Mpc0.93 h/Mpc2.34 h/Mpc

Figure 3.6: Angular dependence of the power spectrum cross-correlation, measured from of 200 density fields, at ki = 0.61,

and kj = 0.14, 0.46, 0.93 and 2.34hMpc−1. The distribution exhibits a correlation of less than 10 per cent for angles larger

than about 30o. For scales of similar sizes, the correlation increases up to 15− 20 per cent for angles smaller than 15o.


3.6.3 From C(ki, kj, θ) to C(ki, kj)

It is possible to recover the covariance matrix one obtains from the angle averaged P (k) by performing

a weighted sum over the angular covariance5. Another test of the accuracy of our method is thus to

compare the C(ki, kj) measured in both ways. This is by far the least convenient way of measuring this

matrix, and we perform this check solely for verification purposes.

We perform this weighted sum and construct C(ki, kj), then compute a similar matrix from our 200

angle averaged power spectra. We present in Fig. 3.7 the cross-correlation coefficient matrix (see [Eq.

3.31]) obtained in the first way, and show the fractional error between both methods in Fig. 3.8. We

observe that they agree at the few per cent level, so long as we are in the non-linear regime. At very

low k-modes, however, many matrix elements are noisy due to the discretization of the shell; the (∆k, θ)

mapping in this coarse grid environment becomes unreliable, and the re-weighting hard to do correctly.

This results in high fractional errors, but at the same time, this region is still in the regime where the

analytic Gaussian prediction is valid. In addition, this paper attempts to solve the bias caused by the

non-Gaussianities that lie in the trans-linear and non-linear regime, in which discretization effects are

much smaller. Finally, we recall that these matrix elements have very little impact on most parameter

studies since such scales contain almost no Fisher information (Rimes & Hamilton, 2005; Ngan et al.,

2012).

3.7 Multipole Decomposition

As shown in last section, we have extracted the power spectrum covariance matrix C(ki, kj , θ), cross-

correlating the 75 different scales selected. Since the final objective is to incorporate this massive object

into generic data analysis pipelines, it must be somehow simplified or made more compact. A quick glance

at the figures of section 3.6 reveals that the angular dependence of the covariance can be decomposed

into a series of Legendre polynomials, in which only a few multipoles will bear a significant contribution.

This allows us to rank the multipoles by importance and to keep only the dominant ones. These results

are further simplified in section 3.8, where we provide fitting formulas to reconstruct C(ki, kj , θ).

In this section, we describe how we perform this spherical harmonic decomposition, then we test our

method on the control samples described in section 3.5, and we finally measure the Cℓ(k, k′) from the

simulations.

5The weights here are simply the number of contribution that enter each angular bin, divided by the square of thetotal number of cells on the k-shell. In other words, because the angular covariance is an average over many pairs of cells,that average must first be undone, then the different angles are summed up, and we finally divide by the total number ofcontributions.


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.7: Cross-correlation coefficient matrix, as measured from integrating the angular covariance. Each matrix

element i, j was obtained from a reweighted sum over C(ki, kj , θ). This is consistent with matrices previously measured in

the literature (Rimes & Hamilton, 2005; Takahashi et al., 2009; Ngan et al., 2012)

0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0.015

0.02

0.025

0.03

0.035

Figure 3.8: Fractional error between the covariance matrices obtained with the two methods. We have suppressed the

largest scales, which are noisy due to low statistics, and present the per cent level agreement at smaller scales. There is a

systematic positive bias of about 1.0 per cent in the calculation obtained from the angular integration, which was detected

in the Gaussian random field test. The 6.0 per cent correlation that was measured has an even smaller impact after the

addition of the zero-lag term.


3.7.1 From C(ki, kj, θ) to Cℓ(ki, kj)

Here we lay down the mathematical relation between C(ki, kj , θ) and Cℓ(ki, kj). Let us first recall that

the spherical harmonics Y ℓm(θ, φ) can serve to project any function F (θ, φ) on to a set of aℓm as:

aℓm =

∫Y ℓm(θ, φ)F (θ, φ)dΩ (3.34)

We substitute F (Ω) → ∆Pi(k) = ∆Pi(k,Ω), which causes the coefficients to be scale dependent, i.e.

aℓm → aℓm(k). The angular power spectrum at a given angular size θ ∼ 1/ℓ is defined as

Cℓ(ki, kj) ≡1

2ℓ+ 1

ℓ∑

m=−ℓ

|aℓm(ki)a∗ℓm(kj)| (3.35)

Combining both equations, and writing Cijℓ ≡ Cℓ(ki, kj) to clarify the notation, we get

Cijℓ =

1

2ℓ+ 1

ℓ∑

m=−ℓ

∫Y ℓm∗(Ω′)Y ℓm(Ω)×

∆P (ki,Ω)∆P∗(kj ,Ω

′)dΩdΩ′ (3.36)

We use the completion rule on spherical harmonics to perform the sum:

ℓ∑

m=−ℓ

Y ℓm(Ω)Y ℓm(Ω′) =2ℓ+ 1

4πPℓ(cosγ) (3.37)

where γ is the angle between the Ω and Ω′ directions, and where Pℓ(x) are the Legendre polynomials of

degree ℓ. We then write

Cijℓ =

1

4π

∫∆P (ki,Ω)∆P

∗(kj ,Ω′)Pℓ(cosγ)dΩdΩ

′ (3.38)

Since we know that ki +∆k = kj, we make a change of variable and rotate the prime coordinate system

such that k always points towards the z-axis. In this new frame, we have dΩ′′ = dcosθ′′dφ′′, where θ′′ is

the angle subtended by ∆k. θ′′ thus corresponds to the angle between the two Fourier modes k and k′.

It is also equal to γ in [Eq. 3.37]. We perform the ‘unprime’ integral first, which gives

Cijℓ =

1

4π

∫Pℓ(cosγ)

∫∆Pi(k)∆Pj(k+∆k)dΩdΩ′′ (3.39)

The inner integral is C(ki, kj , γ), we rename γ → θ and obtain

Cijℓ =

∫Pℓ(cosθ)C(ki, kj , θ)dΩ (3.40)

In practice we are dealing with a discretized grid, hence we must convert the integral of [Eq.3.40]

into a sum. To minimize the error, we use a Legendre-Gauss weighted sum, the details of which can be

found in the Appendix. In order to validate our method, we designed a few tests that are explained in

the following sections.


0

500

1000

1500

2000

Cℓ[M

pc/

h]6 Poisson

0 2 4 6 8 10

0

50

100

150

ℓ

Cℓ[M

pc/

h]6 Gauss

Figure 3.9: (top :) Angular power of the cross-correlation obtained from 200 Poisson densities, at ki∼j ∼ 1.0hMpc−1,

with an offset of one grid cell between the two scales, corresponding to δk = 0.0314hMpc−1. The power at ℓ 6= 0 is consistent

with zero, as expected from [Eq. 3.41]. We recall that the angular dependence of the covariance from Poisson densities

is very weak, hence it projects almost exclusively on the ℓ = 0 term. (bottom :) Gaussian equivalent at ki ∼ 1.0hMpc−1,

and kj = ki +5δk. The analytical prediction is zero at all multipole, while we measure a C0 term of about 80.5h−6Mpc6.

This is caused by the 6 per cent bias we observed in Fig. 3.2.

3.7.2 Testing Cℓ with a Legendre polynomial, with Poisson and Gaussian

distributions

We start our tests by measuring the Cℓ(ki, kj) from the angular dependence of the covariance of power

spectra, which is explicitly set to the second Legendre polynomial on the selected k-shells, as described in

section 3.5.1. We expect the projection to produce a delta function at ℓ = 2, up to numerical precision,

since the Legendre polynomials are mutually orthogonal. We observe from this simple test a sharp peak

at ℓ = 2, which is about two orders of magnitude higher than any other points.

We next measure the Cℓ from the covariance matrix of Poisson densities, whose angular dependence,

we recall, is close to flat (see section 3.5.2), except for the zero-lag point when the two shells are identical.

From the orthogonality of the Legendre polynomials, a flat distribution is projected exclusively on the

first multipole, we thus expect CPoissonℓ (k 6= k′) to peak at ℓ = 0, and to vanish otherwise. Moreover,

we expect the CPoissonℓ (k = k′) to exhibit, in addition, a vertical shift caused by the integration over

the zero-lag point. The analytical expression can be obtained from [Eq. 3.32,3.40]. The azimuthal

integration gives a factor of 2π, the µ delta function gets rid of the last integral, and we get:

CPoissonℓ (k, k′) = 2πCPoisson

22ℓ+1δℓ0 , k 6= k′

= 2πCPoisson2

2ℓ+1δℓ0 + 4πAδkk′ , k = k′ (3.41)


The only scale dependence comes from the surface of the k-shell, and drops as k−2, as explained in

section 3.5.2.

In the k 6= k′ case, we find that in the non-linear regime, the ℓ = 0 point is at least two orders of

magnitude above the other even ℓ, and 18 orders above the odd multipoles. The results are presented in

the top part of Fig. 3.9 for ki∼j ∼ 1.0hMpc−1. The error bars are obtained from a bootstrap resampling.

When k = k′, we find that the zero-lag point effectively shifts the whole distribution upwards by an

amount equivalent to 4πCPoisson(k, k, 0).

Finally, we compare the Cℓ distribution measured from Gaussian fields to the analytical prediction,

obtained from [Eq. 3.40,3.33]:

CGaussℓ (k, k′) = 2π

2〈P (k)〉2N(k)

(1 + (−1)ℓ)δkk′ (3.42)

We measure CGaussℓ from the covariance matrix of 200 Gaussian random fields, as outlined in section

3.5.3. We show the results in the bottom part of Fig. 3.9 for the case where there is a slight offset between

the two scales. Our results are consistent with zero for all multipoles except ℓ = 0, which receives an

undesired contribution from the constant 6 per cent bias described in section 3.5.3 and observed in Fig.

3.2. It turns out that this C0 contribution is very small (i.e. less than one per cent) compared to the

values obtained from simulated density fields, hence we do not attempt to correct for it. In the case

where the two shells are identical, we observe similar results, up to an upward shift caused by the zero-lag

point, which propagates to all multipoles.

3.7.3 Measuring Cℓ(ki, kj) from simulations

We present in this section the multipole decomposition of the C(ki, kj , θ) matrix measured from our

simulations. We show in Fig. 3.10 the first few non-vanishing multipole moments (i.e. ℓ = 0, 2, 4, 6),

in the case where both scales are exactly equal. All the error bars in this section were obtained from

bootstrap resampling. We observe that higher multipoles become closer to the Gaussian prediction

given by [Eq. 3.42], and in fact only the first three differ enough to have a non-negligible impact. As we

progress deeper in the non-linear regime, we expect to encounter a mixture of the following two effects:

an increase in the number of ℓ required to specify the Cℓ distribution, or in the departure from the

Gaussian predictions of a given multipole. As seen from Fig. 3.10, the departure between the multipoles

and the Gaussian power increases for higher k-modes, an effect prominent in the first multipole. The

departure becomes more modest for higher multipoles, and eventually we cannot distinguish between

Gaussian and non-Gaussian. This suggests that the non-Gaussianities are encapsulated in the second of

the effect above mentioned.


0 0.5 1 1.5 2 2.5

100

101

102

103

104

105

k[h/Mpc]

Cℓ(k

,k)[

Mpc/

h]6

C

0

C2

C4

C6

GAUSS

Figure 3.10: Angular power of 200 densities, where ki=j . The dashed line is the Gaussian prediction, obtained from [Eq.

3.42]. From this figure, we observe that the diagonals of multipoles higher than ℓ = 4 converge to the Gaussian predictions.

We then show in Fig. 3.11 the same multipole moments, this time for the case where one scale is

fixed at k = 0.61hMpc−1, while the other is allowed to vary. Once again, higher multipoles have smaller

amplitudes, and approach the null Gaussian prediction. On the diagonal, the relative difference between

the multipoles in the linear regime becomes smaller and converge to the predicted value, as expected. In

addition, in the linear regime, the angular power of the off-diagonal elements (i.e. ki 6= kj) is one to two

orders of magnitude smaller than the diagonal counter part. As we progress to the non-linear regime

however, the off-diagonal elements decrease less rapidly.

3.7.4 Cℓ(k, k′) matrices

In this section, we organize the results into Cℓ(k, k′), and look for the multipole beyond which the

off-diagonal elements become negligible. The whole purpose behind this is to model the full covariance

matrix as:

C(k, k′, θ) =1

4π

∞∑

ℓ=0

(2ℓ+ 1)Cℓ(k, k′)Pℓ(cosθ) (3.43)

where the lower ℓ terms are measured from our simulations, and the others obtained from the Gaussian

analytical prediction ([Eq.3.42]).

In the figures of this section, we present these ‘Cℓ’ matrices, normalized to unity on the diagonal.

These are thus in some sense equivalent to cross-correlation coefficient matrices. Fig. 3.12 presents the

normalized C0 matrix, which shows a structure similar to that of Fig. 3.7. The resemblance is not


0 0.5 1 1.5 2 2.5

0

0.5

1

1.5

k[h/Mpc]

r ℓ(k

,k=

0.6

1h/M

pc)

C

0

C2

C4

C6

Figure 3.11: Same as Fig. 3.10, but with ki = 0.61hMpc−1 being held. The Gaussian prediction is zero in this case. The

measurements are normalized by the square root of their diagonal contributions, such as to show the relative importance

of each multipole. As ℓ increases, the off-diagonal contribution becomes smaller, even for combinations of scales similar in

amplitudes. The fourth point starting from the left is identical to unity for all multipoles, as it corresponds to a diagonal

matrix element.

surprising, since C0 = 4πC(k, k′). This matrix thus contains the information about the error bars of

angle averaged power spectra, as well as their correlation.

By looking at the fractional error between the C0 matrix and the actual covariance matrix of angle

averaged power spectra, we find that our method provides a very good agreement in the trans- and

non-linear regimes, down to the few per cent level (see Fig. 3.13). We do not show the largest scales, in

which our method is more noisy, for reasons already explained. We recall that an extra contribution to

C0(k, k′), not included here, comes from the non-Gaussian Poisson uncertainty, as discussed in section

3.5.2, and needs to be added in the final analysis.

We now present the next few multipole matrices, and find that beyond ℓ = 4, very little information is

contained in the off-diagonal elements. Fig. 3.14 shows the C2 matrix, again normalized to the diagonal

for visual purposes. We observe that the smallest scales are correlated up to 60 per cent.

Fig. 3.15 shows that the correlation in the C4 matrix is still of the order 50 per cent for a good

portion of the non-linear regime. The new feature here is that the strength of the correlation of strongly

non-linear modes among themselves starts to decrease as we move away from the diagonal. Fig. 3.16

shows that C6 is mostly diagonal. As we progress through higher multipole moments, the off-diagonals

become even dimmer, hence do not contain significant amount of new information. From this perspective,


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.12: C0 matrix, normalized such that the diagonal elements are equal to unity. This matrix is completely

equivalent to the cross-correlation coefficient matrix of angle averaged P (k). It represents the correlation between different

scales, and shows that scales smaller than k ∼ 1.0hMpc−1 are correlated by more than 80 per cent.

0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

Figure 3.13: Fractional error between the C0 matrix and that obtained directly from the angle averaged P (k). We do

not show the largest scales, which are noisy due to low statistics and grid discretization. We have also divided the C0

matrix by (4π) for the two objects to match exactly. We recover the 1.0 per cent bias that is seen in Fig. 3.8.


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.14: C2 matrix, normalized such that the diagonal elements are equal to unity. The off-diagonal elements are

still correlated at least at 40 per cent for scales smaller than k = 1.0hMpc−1.

a multipole expansion up to ℓ = 4 is as far as one needs to push in order to model correctly the non-

Gaussian features on the off-diagonal elements.

Following [Eq.3.43], we thus propose to reconstruct the full C(k, k′, θ) from a combination of a) fully

non-linear Cℓ(k, k′) matrices (for ℓ ≤ 4), presented above, b) analytical terms given by [Eq. 3.42] (which

we scale up by 30 per cent as mentioned in section 3.7.3), and c) non-Gaussian Poisson error, which

depends solely on the number density of the sampled fields. In the next section, we decompose and

simplify these Cℓ matrices into a handfull of fitting functions, and show how one can easily reconstruct

the full C(k, k′, θ) at the per cent level precision.

We next present in Fig. 3.17 the ratio of the diagonal of these matrices to the Gaussian prediction.

We observe that all of them are consistent with the prediction in the linear regime. As we progress

towards the non-linear regime, the largest departure comes from the C0 matrix, by a factor of about

40 near k = 1.0hMpc−1. We observe a turn over at smaller scales, which is caused by our resolution

limit. We opted not to model it in our fitting formula. C2 and C4 mildly break away from Gaussianity

by factors of 4 and 2 at the same scale. All the higher ℓ’s are consistent with Gaussian statistics.

Over-plotted on the figure are fitting formulas, which are summarized in Table 3.1.


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.15: C4 matrix, normalized such that the diagonal elements are equal to unity. The off-diagonal elements close

to the diagonal are correlated at the 30 per cent level in the non-linear regime.

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Figure 3.16: C6 matrix, normalized such that the diagonal elements are equal to unity. We observe that the matrix is

mostly diagonal, and thus decide to treat C6 and all higher multipoles as purely Gaussian.

Table 3.1: Fitting formulas for the ratio between the diagonals of the Cℓ(k, k′) and the Gaussian prediction. For all ℓ’s,

the function is modelled by V (x) = 1.0 + (x/α)β .

ℓ α β

0 0.2095 1.9980

2 0.5481 0.7224

4 1.6025 1.0674


10−1

100

100

101

102

k[h/Mpc]

Ck,k

ℓ/C

gauss

ℓ

ℓ = 0

ℓ = 2

ℓ = 4

ℓ = 6

Figure 3.17: Ratio of the diagonal elements of a few Cℓ matrices, compared to the Gaussian prediction. The error bars

were obtained from bootstrap resampling. Over-plotted are the fitting functions summarized in Table 3.1.

3.8 Factorization of the Cℓ Matrices

In this section, we simplify even further our results with an Eigenvalue decomposition of the normalized

Cℓ(k, k′) matrices, as shown in the figures of section 3.7.4. We perform an iterative process to factorize

each matrix into a purely diagonal component and a symmetric, relatively smooth off-diagonal part.

The later can be further decomposed into a small set of Eigenvectors Uλ(k), corresponding to the largest

Eigenvalues λ. These are then fitted with simple formulas. Combined with Gaussian predictions and

fitting formulas for the diagonal, one can fully reconstruct each of the Cℓ(k, k′) matrix, and thus recover

C(k, k′, θ) as well.

We start off the iteration by assigning the identity matrix to the diagonal component, which we

subtract from the original matrix. We then extract from the remainder the principal Eigenvectors and

recompose a new matrix as

rℓ(k, k′) ≡ Cℓ(k, k

′)√Cℓ(k, k)Cℓ(k′, k′)

= δkk′ +∑

λ

λUλ(k)Uλ(k′) (3.44)

For the next iterations, we model the diagonal as δkk′ −∑λ λU2λ(k), and decompose the remainder

once again. We iterate until the results converge, which takes about 4 steps. We vary the number of

Eigenvalues in our iterations, and keep the minimal number for which the reconstruction converges. In

the end, the rℓ(k, k′) matrix is modelled as:

rℓ(k, k′) = δkk′

[1− λU2

λ(k)]+∑

λ

λUλ(k)Uλ(k′) (3.45)


0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

−0.025

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

Figure 3.18: Fractional error between the original C0 matrix and that produced with the principal Eigenvector. We do

not plot the largest scales, which are noisy due to low statistics and grid discretization.

We show in Fig. 3.18 the fractional error between the original matrix and the factorized one. The

factorization of the C0 matrix with one Eigenvector reproduces the original matrix at the few per cent

level. The same procedure is also applied for the higher multipoles, in which we have included the

first four Eigenmodes, and we find that the fractional error between the reconstructed and the original

matrices are also of the order of a few per cent.

We next fit these Eigenvectors with simple functions: for all ℓ’s, the first Eigenvector is parameterized

as U(k) = α(

βk+ γ)−δ

, and all the other vectors as U(k) = αkβsin(γkδ). The values of (α, β, γ, δ) for

the lowest three ℓ’s are presented in Table 3.2. We require that all these formulas vanish as k → 0, since

the Cℓ matrices become diagonal in the linear regime. The Eigenvectors of the C4 matrix are presented

in Fig. 3.20; over-plotted are the fitting formulas. The pixel-by-pixel agreement between the original

matrices and those obtained from the fitted formulas is within less than 10 per cent for k > 0.5.

Larger scales fluctuate much more as they are less accurately measured, hence the pixel-by-pixel

agreement is not expected there. In addition, the matrices with ℓ ≥ 6 are much harder to express with

a small set of Eigenvectors, since the Eigenvalues are not decreasing fast enough. In any case, the first

three harmonics we provide here contain most likely all the information one will ever use in realistic

surveys and forecasts.


0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

Figure 3.19: Fractional error between the original C0 matrix and that produced with the fitting formulas. We do not

show the largest scales, which are noisy due to low statistics. The per cent level bias that was seen previously in Fig. 3.8

is no longer obvious, as the main source of error now comes from fitting the Eigenvector.

0 0.5 1 1.5 2 2.5−0.15

−0.1

−0.05

0

0.05

0.1

0.15

0.2

k[h/Mpc]

Uλ(k

)[M

pc/

h]3

λ = 1

λ = 2

λ = 4

λ = 3

Figure 3.20: Four principal Eigenvectors of the normalized C4 matrix (solid lines), and corresponding fitting formulas

(dotted lines).


Table 3.2: Fitting parameters for the Eigenvectors of the Cℓ matrices, with their corresponding Eigenvalues. For all ℓ’s,

the first Eigenvector is parameterized as U(k) = α(

βk+ γ

)−δ, and all the other vectors as U(k) = αkβsin(γkδ). These

parameters were obtained from dark matter N-body simulations, but the method is general, and a different prescription

of galaxy population may result in slightly different values.

ℓ λ α β γ δ

0 61.9058 0.0501 0.0207 0.6614 2.3045

2 35.7400 0.273 0.8266 1.962 0.816

4.4144 0.15772 2.4207 0.79153 0.032207

1.7198 0.14426 4.0613 0.76611 -0.26272

0.9997 0.14414 5.422 0.84826 0.31324

4 22.0881 0.060399 0.10344 0.64008 2.2584

4.5984 0.1553 2.3370 0.9307 -0.1154

2.2025 0.1569 3.6937 0.92304 0.04006

1.4062 0.15233 5.1617 0.8899 -0.14503

3.8.1 Non-Gaussian Poisson noise

The non-Gaussian Poisson uncertainty, whose construction was presented in section 3.5.2, can conve-

niently be incorporated in an analysis by finding the principal Eigenvalue and Eigenvector of CPoisson0 (k, k′).

Higher multipoles are not relevant as the angular distribution is flat, as shown in the middle plot of Fig.

3.2. We test three number densities, corresponding to n = 5.0×10−5, 1.52×10−4 and 1.0×10−2h3Mpc−3.

In all cases, we decompose the covariance matrix into a diagonal component and a cross-correlation

coefficient matrix, find the matrix’s principal Eigenvalue and Eigenvector, then fit the latter with:

UPoissonfit (k) = α

(βk+ γ)−δ

. The diagonal is also fitted with a simple power law of the form

V Poisson(k) ≡ CPoisson(k, k)

CPoissonGauss (k, k)

= eǫkσ (3.46)

where CPoissonGauss (k, k) ≡ P 2

Poisson(k)

N(k) . The best-fitting parameters are summarized in Table 3.3, and the

performance of the Eigenvector fit can be found in the Appendix.

3.8.2 Recipe

Here we summarize our method to generate accurate non-Gaussian covariance matrices. The full

C(k, k′, θ) matrix is then given by [Eq. 3.43], where the ℓ ≤ 4 terms are obtained from the fitting

functions, and the higher multipole moments are obtained directly from [Eq. 3.42]. The sum over these


Table 3.3: Fitting parameters for the diagonal of the CPoisson0 (k, k′) matrix, and for the principal Eigenvector of the

cross-correlation coefficient matrix. For all three number densities (i.e. n1,2,3 = 5.0 × 10−5, 1.52 × 10−4 and 1.0 × 10−2

respectively), the Eigenvector is parameterized as UPoissonfit (k) = α

(

βk+ γ

)−δ, and the ratio of the diagonal to the

Gaussian prediction is fitted with V Poisson(k) = eǫkσ . Top to bottom rows correspond to increasing density.

λ α β γ δ ǫ σ

52.02 1.0193 0.0947 2.1021 2.5861 2.6936 2.1347

45.09 0.9987 0.2034 2.1553 2.3407 1.6533 2.1965

24.41 0.2966 3.3736 0.6099 0.6255 -0.4321 2.0347

Gaussian terms can be evaluated analytically as

1

2

∞∑

ℓ=6

(2ℓ+ 1)(1 + (−1)ℓ)Pℓ(µ) = δD(1 + µ) + δD(1− µ)− 1− 5P2(µ)− 9P4(µ) (3.47)

For the non-Gaussian terms, we proceed as follow: each of the normalized Cℓ(k, k′) can be constructed

from the first set of fit functions Uλ(k) provided in Table 3.2, and following [Eq. 3.45]. The ‘un-

normalized’ Cℓ(k, k′) terms are then constructed by inverting [Eq. 3.31], where the diagonal elements

are obtained from the product of the Vℓ(k), also summarized in Table 3.2. The Gaussian prediction is

obtained from [Eq. 3.42]. In other words:

Cℓ(k, k′) =

(δkk′

(1−

∑

λ

λU2λ,ℓ(k)

)+∑

λ

λUλ,ℓ(k)Uλ,ℓ(k′)

)×

√Vℓ(k)Vℓ(k′)CGauss

ℓ (k)CGaussℓ (k′) (3.48)

The complete covariance matrix is given by:

C(k, k′, µ) =1

4π

3∑

ℓ=0

(2ℓ+ 1)Cℓ(k, k′)Pℓ(µ)+

2P (k)2

N(k)

(δD(1 + µ) + δD(1− µ)− 1− 5P2(µ)− 9P4(µ)

)(3.49)

with µ = cos(θ). This can be written in a more compact form as

C(k, k′, µ) = CGauss(k)δ(k − k′) +3∑

ℓ=0

(2ℓ+ 1)

(Gℓ(k)δ(k − k′) +Hℓ(k, k

′)Pℓ(µ)

)(3.50)

with

Gℓ(k) = CGauss(k)(Vℓ(k)− 1) (3.51)


Hℓ(k) =∑

λ

(Fλ,ℓ(k)Fλ,ℓ(k

′)− F 2λ,ℓ(k)δ(k − k′)

)(3.52)

and

Fλ,ℓ(k) = Uλ,ℓ(k)√λVℓ(k)CGauss(k) (3.53)

We conclude this section with a word of caution when using the fitting formulas provided here,

in the sense that the range of validity of the fit has not been tested on other cosmological volumes.

Consequently, we advice that one should limits itself to k ≤ 2.0hMpc−1.

3.9 Measuring the Impact with Selection Functions

This section serves as a toy model for a realistic non-Gaussian error analysis, as it incorporates the

non-Gaussian covariance matrix measured from N-body simulations with the 2dFGRS selection function

(Norberg et al., 2002). We compare the estimated error bars on P (k) between the naive, purely diagonal,

Gaussian covariance matrix, the effect of the one-dimensional window function as prescribed by the FKP

formalism, the unconvolved non-Gaussian covariance as measured from our 200 N-body simulations, and

the convolved non-Gaussian matrix6.

We recall that in a periodic volume, a top hat selection function makes the observed and underlying

covariance matrices identical. That only occurs in simulated volumes, and in that case, no convolution

is necessary. Non-periodicity is dealt with by zero-padding the observed survey, and already results in

some coupling between different power spectrum bands. The coupling becomes more important as the

selection function departs from a top hat, and in that case, the best estimator of the observed covariance

matrix is a convolution of the 6-dimensional covariance over both vectors (k,k′), given by:

Cobs(k,k′) =

∑k′′,k′′′ Ctrue(k

′′,k′′′)|W (k− k′′)|2|W (k′ − k′′′)|2

(N2Nc

∑xW

2(x)w2(x))2(3.54)

The denominator is straightforward to calculate, while the numerator is a 6-dimensional integral, which

must be calculated at all of the 6-dimensional coordinates, a task computationally impossible to perform.

For example, with n3 cells on the grid, we have to sum over n6 terms for each (k,k′) pair. There are n6

such pairs, and each term takes about 3 flop of computation time. For n = 100, this process would take

3 ∗ 1024 flop, and current supercomputers are in the regime of resolving 1012 flop per seconds. The

above calculation would therefore take about 3000 years to complete. With the factorization proposed

in this work however, we break down the computation into smaller pieces and reduce the dimensions to

seven at most.

6The code that was used to perform these calculations is made available onwww.cita.utoronto.ca/∼jharno/AngularCovariance/, and additional explanations can be provided upon request.


Table 3.4: List of weights w(θ, φ) needed for the angular integrals over the selection function. These can be precomputed

to speed up the convolution. All integrals are in the form of [Eq. 3.56].

cos2(θ) sin2(θ) cos2(θ)e±2iφ sin2(2θ)e±iφ

cos4(θ) sin4(θ) sin4(θ)e±2iφ sin4(2θ)e±4iφ

sin2(2θ) sin2(2θ)e±2iφ sin(θ)cos3(θ)e±iφ cos(θ)sin3(θ)e±iφ

3.9.1 Factorization of the 6-dimensional covariance matrix

We break down the true covariance matrix C(k′′,k′′′) into a product of simple functions of the form

Hℓ(k′′), Gℓ(k

′′) and Pℓ(µ), where the angular components come exclusively from the Legendre polyno-

mials. Again, µ is the (cosine of the ) angle between k′′ and k′′′, and must first be expressed in terms of

(θ′′, φ′′, θ′′′, φ′′′), following [Eq. 3.14]7. The only multipoles that appear in our equations are ℓ = 0, 2, 4,

so µ is to be expanded at most up to the fourth power. For a full factorization, the terms including

cos(φ′′ − φ′′′) must further be re-casted in their exponential form with Euler’s identities.

When computing the convolution, the first term on the right hand side of [Eq. 3.50] is spherically

symmetric, hence it must be convolved with the selection function as:

CobsGauss(k, k

′) =∑

k′′

CGauss(k′′)|W (k′′ − k)|2|W (k′′ − k′)|2 (3.55)

which is pretty much the FKP prescription, namely that the selection function is the only cause of mode

coupling.

For the other (i.e. non-Gaussian) terms of [Eq. 3.50], we use the fact that the only coupling between

the k′′ and k′′′ vectors comes from the delta function, which couples solely their radial components.

This means that all the angular integrations can be precomputed and stored in memory. For example,

the only angular dependence in the ℓ = 0 multipole comes from the selection function itself, hence we

can precompute

X(k, k′′) =∑

θ′′,φ′′

|W (k − k′′)|2sin(θ′′)w(θ′′, φ′′) (3.56)

and the convolution is now four dimension smaller. The weight function w(θ′′, φ′′) is equal to unity for

the C0 term, and the sin(θ′′) comes in from the Jacobian in angular integration. For the other multipoles,

more terms must be precomputed as well, whose weight functions are summarized in Table 3.4.

7In this section, we use µ instead of cos(θ) to denoted the (cosine of the) angle between the two Fourier modes, to avoidconfusion with θ′ and θ′′′, which corresponds to the angle of (k′′,k′′′) with respect to the x-axis.


3.9.2 The 2dFGRS selection function

The 2dFGRS(Colless et al., 2003) is comprised of two major regions, the NGP and the SGP, each of which

takes the overall form of a fan of 75× 5 degrees, extending from z = 0.02 to 0.22. The selection function

is constructed by first integrating the luminosity function dΦ(L)/dL over all the observed luminosity

range, which is both redshift and angle dependent. The results need to be multiplied by the redshift

completion function R(θ, φ). The parameters that enter this calculations (Φ⋆, α and M⋆ − 5log10h) are

obtained from the 2dFGRS as −1.21, 1.61 × 10−2h3Mpc−3 and −19.66 respectively. The two angular

maps (R(θ, φ) and bJ(θ, φ)) required are publicly available on the 2dFGRS website8. It is possible to

obtain an even more accurate selection function by taking into account the redshift dependence of the

magnitude sensitivity, however we do not need such an accuracy for the current work. Finally, our

selection function is normalized such that

∫|W (k)|2d3k = 1 (3.57)

To understand the impact of the non-Gaussian Poisson uncertainty on the measured uncertainty,

we test various templates, keeping the 2dFGRS selection function fixed. We follow the procedure of

section 3.5.2, with an average number density of ngal = 1.52× 10−4h3Mpc−3, which corresponds to an

early data release of the 2dFGRS data. The final release contains more objects, and has a density of

about n = 5.0 × 10−2h3Mpc−3. By comparison, the Poisson uncertainty corresponding to the number

count of the Wiggle-Z survey could be modelled with n = 5.0×10−5h3Mpc−3 for partial data and about

2.0×10−4h3Mpc−3 for the final data release. We thus opt for two more number densities: n = 1.52×10−4

and n = 1.0× 10−2.

3.9.3 Results

We assign the selection function on to a 256x256x128 grid, where the lower resolution is along the

direction perpendicular from the NGP. We precompute the Fourier transform, W (k) and square each

terms. Fig. 3.21 shows a comparison between the angle average of |W (k)|2 and a fitting function

provided by the 2dFGRS group.

We then define a second set of bins in spherical coordinates, over which we perform the convolution.

For that purpose, we divide the original volume of the survey into 64 radial bins, 48 polar bins and 32

azimuthal bins. The selection function is assigned on the grid by averaging over the 27 closest cells in

the original grid. We have included a sin(θ) terms in each integrals over the polar angle, and a k2 in

8www.mso.anu.edu.au/2dFGRS/


10−2

10−1

100

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

k[h/Mpc]

|W(k

)|2

2dFGRS Selection FunctionApproximate Fit

Figure 3.21: The angle average of the 2dFGRS selection function, compared to an approximate fit provided by Percival

et al. (2001). The fit is not perfect as it was obtained with an earlier estimate of the selection function. We also note that

our method differs in details with that used in Cole et al. (2005) by the fact that we imposed a cut at redshift of z = 0.22,

and that we used a somewhat lower resolution.

each radial integral to properly account for the Jacobian matrix in spherical coordinates.

Fig. 3.22 shows the diagonal of the convolved covariance matrix, divided by P 2(k), for the FKP

prescription and for the progressive inclusion of ℓ = 0, 2 and 4 multipoles. Also overploted is the non-

Gaussian results without the convolution. We see that already at k ∼ 0.1hMpc−1, the non-Gaussian

fractional error, after the convolution, deviates from the FKP prescription by a factor of about 3.0,

while the unconvolved C0 still traces quite well the FKP curve. This means that the mode mixing

caused by the convolution with the survey selection function increases significantly the variance of the

observed power spectrum. The departure gets amplified as one progresses towards higher k−modes,

and, by k ∼ 1.0hMpc, the unconvolved C0 departs from the FKP prescription by almost two orders of

magnitudes. Interestingly, the convolved C0 merges with the unconvolved counterpart at k ∼ 0.5, where

the BAO scale is usually cut off. Inclusion of higher multipole increases the variance by a factor of about

2.0. We have overplotted a simple smooth fitting function of the form :

Cfit(k) = Cg(k)(1 +

2.3

(0.08/k)3.7 + (0.08/k)1.1+ 0.0007

)(3.58)

which approximates the contribution from the three lower multipoles.

Fig. 3.23 shows the convolved cross-correlation coefficient matrix, where the angle average has been

taken after the convolution. It is also possible to factorize this matrix, hence we proceed to an Eigenvalue

decomposition, following the same iterative procedure as in section 3.8, solving for the first Eigenvector

only. The Eigenvalue was found to be λ = 19.7833, and we used the sum of a quadratic and a Gaussian


10−1

100

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

k[h/Mpc]

(σP

/P

)2

Gauss

n1

n2

n3

GaussUnconvolved C0C0C0 + C2C0 + C2 + C4Fit

Figure 3.22: Diagonal of the convolved covariance matrix, first with no multipole, i.e. following FKP prescription (thick

dashed line), then with the progressive inclusion of the C0 (open circles), the C2 (solid points) and the C4 multipoles

(stars). Also shown is the diagonal of the unconvolved C0 terms directly measured from N-body simulations (thick solid

line), and a fitting function for the total covariance (thin solid line). Finally, the inclusion of the non-Gaussian Poisson

noise is represented by three dotted lines, representing the three number density detailed in Table 3.3. The 2dFGRS final

data release has a number density of the order 5.0× 10−2h3Mpc−3, which thus lies between n2 and n3.

function to model the Eigenvector:

Uobsλ (k) = Aexp[− 1

σ2log2 (k/kp)] + (alog2 (k/ko) + blog (k/ko) + c) (3.59)

with A = 0.1233, σ = 1.299, a = 0.0049, b = 0.0042, c = 0.0052 and (kp, ko) = (0.17, 0.008)hMpc−1

respectively. A comparison of the fit and the actual vector is presented in Fig. 3.24. The noise reduced

cross-correlation coefficient matrix is presented in Fig. 3.25. We observe that the Fourier modes are

already more than 50 per cent correlated at k = 0.1hMpc−1, a significant enhancement compared to the

unconvolved C0 matrix, in which the equivalent coupling occurs roughly towards k = 0.22hMpc−1. This

would most likely have an impact on a non-Gaussian BAO analysis.


10−2

10−1

100

10−2

10−1

100

k[h/Mpc]

k[h

/M

pc]

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.23: Normalized convoluted covariance matrix with all three multipole.

10−2

10−1

100

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

k[h/Mpc]

U0(k

)

EigenvectorFit

Figure 3.24: Principal Eigenvector of the convolved C0 matrix, compared to a simple fitting formula. The fractional

error of the fitting function is at most 13 per cent.


10−2

10−1

100

10−2

10−1

100

k[h/Mpc]

k[h

/M

pc]

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3.25: Normalized convoluted covariance matrix with all three multipole, reconstructed from a fit of the principal

Eigenvector.

3.10 Discussion

Generally speaking, the departures from Gaussianity will be sensitive to the survey parameters. The

quantitative results presented here apply only to the 2dFGRS, however similar calculations could be

carried for other surveys. We have found that even for modes of k ∼ 0.1hMpc−1, the non-Gaussian error

bars are higher than those prescribed by the FKP method by a factor of a few, due to mode coupling

caused by the convolution of the selection function. This has to be put in contrast with results from pure

N-body simulations, which show that the departure from Gaussianity reaches this sort of amplitudes

at higher k-modes, as seen from Fig. 3.22. We also observe that with the 2dFGRS, the non-Gaussian

Poisson noise plays an important role if the number density is smaller than 0.01h3Mpc−3, but is not

enough to characterize all of the non-Gaussian features of the density field. The C0 term is the leading

contribution of the enhancement observed in the range k = 0.06− 0.4hMpc−1, but for larger k-modes,

C2 and C4 both play an important role.

In the absence of a survey selection function, significant changes in the covariance matrix do not

necessarily translate into noticeable changes in the BAO constraints. For instance, assuming that the

BAO mean measurement was performed with a non-Gaussian estimator, the propagation of the non-

Gaussian error on to the dilation scale produces constraints that are hardly distinguishable from the

naive Gaussian treatment (Takahashi et al., 2011). In the data analyses however, the estimators of the

mean are usually Gaussian, while the power spectrum covariance matrices that enter the calculations

are either Gaussian or obtained with mock catalogues. As pointed out previously (Ngan et al., 2012),


the estimators constructed in such a way are inconsistent and should be recalculated to include noise in

the measured mean. It was found that the corrected – i.e. consistent – error bars are about 10 per cent

higher.

It is worth mentioning again that the angle integration of C(k, k′, θ) provides an alternative way to

extract the covariance matrix of the angle average power spectra C(k, k′). Although the mean value of

both methods is identical, i.e. unbiased, the second gives us a better handle on the error on each matrix

element, hence provides an optimal measurement of their uncertainty. We have shown in this paper that

each matrix element receives its dominant contribution from small angles, while larger angles are noisier.

It is thus possible to re-weight the angular integration by taking this new information into account, and

obtain more accurate error bars on each matrix element, compared to the current bootstrap method.

As mentioned in the introduction, our next objective in HDP2 is to achieve a similar accuracy with a

much lower number of simulations. This would revolutionize the field of observational cosmology as the

covariance matrix could be measured internally, i.e. directly from the data.

The techniques presented in this paper call for extensions, as we did not include redshift distortions in

our analysis. Also, shot noise will become important when repeating this procedure on haloes, motivated

by recent finding that the Fisher information in haloes is also departing from Gaussianity (Neyrinck et al.,

2006). It is straightforward to perform a similar analysis with a quadratic halo model, where the halo

density is parameterized by δhalo(x) = Aδ(x)+Bδ2(x). This involves an extra cross-correlation between

the linear and quadratic term, and leaves some room for the choice of A and B, and ultimately, one

should work straight from a halo catalogue. The optimal estimator should also be based on a model that

is cosmology independent, hence one should compute how the fitting functions scale with Ωm, ω and z.

As mentioned earlier, the effect of the selection function is enhanced for survey geometries that are

different from top-hats, and it would be interesting to repeat some of the BAO data analyses that were

performed on such surveys, like the 2dFGRS or Wiggle-Z. The current method also applies to surveys

with irregular geometries like those obtained from the Lyman-α forest (McDonald & Eisenstein, 2007;

McQuinn & White, 2011), and we are hoping it will be considered in the elaboration of these future

analysis pipelines. In addition, the extraction of non-Gaussian error bars from two-dimensional angular

clustering could also be performed with techniques similar to those employed here. We leave it for

future work to match our results with predictions from higher order perturbation theory. We would like

to verify that the angular dependence we observe in the covariance matrix is predicted by a complete

4-points function analysis, at least in the trans-linear regime.

The results presented in section 3.7.4 and the recipe presented in the one preceeding can find useful

applications in the field of weak lensing. Convergence maps, for instance, are constructed from a redshift


integral over a past line cone filled with dark matter, weighted by a geometric kernel. Because of the

projection nature of this process, the survey maps are sensitive to both large and small scales, where

non-Gaussianities have been observed in the convergence power spectrum (Dore et al., 2009).

The lensing fields are quadratic functions of smoothed temperature fields, and the optimal smoothing

window function depends not only on the parameter under study, but also on the statistical nature of the

sources and lenses (Lu & Pen, 2008). Optimal quadratic estimators of lensing fields were first obtained

under the Gaussian assumption (Hu & Okamoto, 2002; Zahn & Zaldarriaga, 2006), then from N-body

simulations (Lu et al., 2010), where it was found that the optimal smoothing window function for dark

energy involves the first two multipoles of the dark matter power spectrum covariance matrix, C0(k, k′)

and C2(k, k′) (see [Eq. 23 − 24] in (Lu et al., 2010)), even in absence of survey selection function. The

tools developed in the present paper thus allow one to construct, for the first time and from simple

fitting functions, optimal non-Gaussian estimators of dark energy parameters from 21 cm temperature

maps.

3.11 Conclusion

Estimating accurately the non-linear covariance matrix of the matter power spectrum is essential when

constraining cosmological parameters including, but not restricted to, the dark energy equation of state

ω. So far, many BAO analyses from galaxy surveys were performed under the assumption that the

underlying density field is Gaussian, which yields a suboptimal measurement of the mean power spectrum

and thus of the BAO dilation scale. In addition, and at least as important, the estimated error bars are

biased.

To estimate unbiased error bars on the dilation scale is a challenging task, but can now be done. In

the simple case of periodic volume, it was shown recently (Ngan et al., 2012) that, first, an unbiased error

bar on a suboptimal measurement of the mean could be obtained from the knowledge of the underlying

covariance matrix. Second, if one did measure optimally the mean BAO dilation scale, then the optimal

measurement of the error requires an estimate of the inverse of the power spectrum covariance matrix.

This is much more challenging due to the presence of noise, even when dealing with simulations embedded

in periodic volumes, but improves the constraining performance by a significant amount.

When estimating the power spectrum and its uncertainty from data, the calculations are more in-

volving since all observed quantities are actually convolved with the survey selection function. The

covariance matrix is not isotropic, as it depends on the relative angle between two Fourier modes, hence

the convolution cannot be simply factored into two radial components. We are left with a challenging


six-dimensional integral to perform, which so far has been an unresolved problem.

In this paper, we present a method to perform this convolution for an arbitrary galaxy survey

selection function, and thus allows one to measure unbiased error bars on the matter power spectrum.

The estimate is still suboptimal, unless one combines our tools with the PKL formalism. From an

ensemble of 200 N-body simulations, we have measured the angular dependence of the covariance of the

matter density power spectrum. We have found that on large scales, there is only a weak dependence,

consistent with the Gaussian nature of the fields in that regime. On smaller scales, however, we have

detected a strong signal coming from Fourier modes separated by small angles. This comes from the fact

that the complex phases of these modes are similar, hence they tend to couple first. We next expanded

the covariance C(k, k′, θ) into a multipole series, and found that only the first three even poles were

significantly different from the Gaussian calculations. We further decomposed these Cℓ(k, k′) matrices

into diagonal terms and cross-correlation coefficient matrices, from which we extracted the principal

Eigenvectors. This allowed us to break down the underlying covariance into a set of Eigenvectors,

Eigenvalues plus three diagonal terms. We provided simple fitting formulas for each of these quantities,

and thus enable one to construct a full six-dimensional covariance matrix with an accuracy at the few

per cent level.

Intrinsically, non-Gaussianities introduce N2 matrix elements to be measured from N-body simu-

lations, as opposed to N for Gaussian fields. With the proposed method, the number of parameters

to measure is reduced to a handful, even if the survey selection function is non-trivial. This factor-

ization is necessary in order to estimate unbiased non-Gaussian error bars on a realistic galaxy survey.

We found that in the case of the 2dFGRS selection function, the non-Gaussian fractional variance at

k ∼ 0.1hMpc−1 is larger by a factor of three compared to the estimate from the FKP prescription,

and by more than an order of magnitude at k ∼ 0.4hMpc−1. With similar techniques, we were able

to propagate a few templates of non-Gaussian Poisson error matrices into the convolution and estimate

the impact on the measured power spectrum. We showed that with the 2dFGRS selection function, the

non-Gaussian Poisson noise corresponding to a number density significantly lower than 0.1h3Mpc−3 has

a large effect on the fractional variance at scales relevant for BAO analyses and should be incorporated

in an unbiased analysis.

The cross-correlation coefficient matrix of the convolved power spectrum shows that the correlation

propagates to larger scales in the convolution process, and should have a larger impact on BAO analyses

for instance. We conclude by emphasizing on the fact that constraints on cosmological parameters

obtained from BAO analyses of galaxy surveys are currently significantly biased and suboptimal, but

that both of these effects can now be dealt with.


Acknowledgments

The authors would like to thank Olivier Dore, Ilana MacDonald and Patrick McDonald for useful dis-

cussions, Daniel Eisenstein, Martin White and Chris Blake for their comments on early versions of the

manuscript, and to acknowledge the financial support provided by NSERC and FQRNT. The simulations

and computations were performed on the Sunnyvale cluster at CITA. We also acknowledge the 2dFGRS

mask software written by Peder Norberg and Shaun Cole.

3.A Legendre-Gauss Weighted Summation

The conversion of the integral into a sum is performed using a Legendre-Gauss weighted sum (Abbott,

2005), in which ℓ ‘collocation’ knots, labeled µk with k = 1, 2, . . . ℓ, are placed at the zeros of the Legendre

polynomial Pℓ(µ). We choose ℓ = 101, and we exclude the end points at µ = ±1 in order to isolate the

zero-lag contribution. The weights wk are given by:

wk =2

(1− µ2k)(dPℓ=101/dµ(µk))2

(3.60)

This Gaussian quadrature gives an exact representation of the integral for polynomials of degree 201 or

less, and provides a pretty good fit to most of our C(ki, kj , θ). In the linear regime, the discretization

effect becomes important, and the number of angles one can make between the grid cells drops down

as k2. In the case were fewer points are available, we choose ℓ = 51, 21, 11 or 5, depending on the

number of available angular bins. Once we have specified the knots, then, for each scale combination, we

interpolate the angular covariance on to these knots, and then perform the weighted sum. As mentioned

above, we always treat the zero-lag point separately in order to avoid interpolating its value to the

nearest neighbours. We thus break the summation in two pieces:

Cijℓ = 2π

∑

µk 6=±1

Pℓ(µk)C(ki, kj , µk)wk + 2πC(ki, kj , µ = 1)∆µ(1 + (−1)ℓ) (3.61)

The factor of 2π comes from the integral over the φ angle, and ∆µ is half the distance to the first knot.

3.B Eigenvector of the Poisson noise

This Appendix presents the Eigenvector that best describes the non-Gaussian Poisson noise, as discussed

in section 3.8.1. We restrict ourselves to the case where the number density is the highest, even though

similar analyses can be carried for other values of n studied in this paper. We present in Fig. 3.26 the

Eigenvector itself, along with the best-fitting formula provided. We next compare the covariance matrix


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

k[h/Mpc]

UP

ois

son

0(k

)

EigenvectorFit

Figure 3.26: Principal Eigenvector of the cross-correlation coefficient matrix associated with the non-Gaussian Poisson

noise, compare to our best-fitting formula.

constructed from the fitting functions with the original, and present the fractional error in Fig. 3.27,

which shows an agreement at the per cent level. When compared with the predictions from Cohn (2006),

we observe that the overall trends are consistent: first, the Gaussian contribution to the error decreases as

one probes smaller scales. Second, densities with lower n see their Gaussian contribution being reduced

in the trans-linear regime, where the non-Gaussian Poisson counting becomes more important. Third,

densities with lower n produce larger cross-correlation coefficients between trans-linear scales.


1 1.2 1.4 1.6 1.8 2

1

1.2

1.4

1.6

1.8

2

k[h/Mpc]

k[h

/M

pc]

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

Figure 3.27: Fractional error between the original cross-correlation coefficient matrix associated with the non-Gaussian

Poisson noise, and that constructed with our best-fitting functions. We have not shown the lowest k-modes since these are

very noisy.

Chapter 4

Non-Gaussian Error Bars in Galaxy

Surveys-II

4.1 Summary

We propose a novel method that aims at estimating non-Gaussian error bars on the matter power

spectrum directly from galaxy survey data. We utilize known symmetries of the 4 point function to

estimate the power spectrum covariance matrix with a minimum number of independent fields. Even

though the results are noisy, the underlying matrix can be mostly recovered with three straightforward

noise-reductions techniques. We find that 4 fields are enough to achieve a convergence on the full

covariance matrix that is comparable to the standard brute force errors obtained with 200 simulations.

We further parameterize the non-Gaussian features following a prescription given by the first paper in

this series, and show how to extract the principal parameters from only 4 fields. We describe a handful

of noise-reduction techniques that improve the signal, and assess the quality of the extracted covariance

matrix with Fisher information measurements. We provide error bars on the parameters and the Fisher

information whenever possible, and show that we are able to estimate the Fisher information to within

20 per cent up to k = 1.0hMpc−1. The improvement is significant since, in comparison, the Gaussian

estimate is off by up to two orders of magnitude there.

93

Chapter 4. Non-Gaussian Error Bars in Galaxy Surveys-II 94

4.2 Introduction

With current (York et al., 2000; Colless et al., 2003; Schlegel et al., 2009b; Drinkwater et al., 2010) and

future (LSST Science Collaborations et al. (2009); Benıtez et al. (2009), Pan-STARRS1, DES2) galaxy

surveys, measuring the matter power spectrum with percent level precision has become one of the main

task of modern cosmology. Although its amplitude contains information about a number of cosmological

parameters, it is its sensitivity to the dark energy equation of state that motivates the community to

perform this challenging measurement (see Albrecht et al. (2006) and references therein). More precisely,

it is through a measurement of the Baryonic Acoustic Oscillation (BAO) dilation scale that information

about dark energy is extracted from galaxy surveys (Eisenstein et al., 2005; Hutsi, 2006; Tegmark et al.,

2006; Percival et al., 2007; Blake et al., 2011; Anderson et al., 2012).

One of the challenge that each of these measurements has to face is to estimate the correlated

uncertainty on the matter power spectrum. The dynamical range of the BAO signal is overlapping with

both the linear and non-linear regimes, as a consequence it is sensitive to the non-Gaussian features of

the underlying matter field. It was recently shown that the error bars in the power spectrum are strongly

correlated as one progresses in the non-linear regime (Meiksin & White, 1999; Rimes & Hamilton, 2005;

Takahashi et al., 2009). Early BAO treatments avoided these complications by excluding most of the

non-linear scales (Seo & Eisenstein, 2003), but at the same time, the highest statistical accuracy exists

at these small scales, where many Fourier modes contribute to a given power spectrum bin. Optimal

analyses must therefore attempt to characterize the non-Gaussian nature of the measured fields, and

construct their estimators based on these results.

The impact of non-Gaussian features on measurements of the BAO dilation scale is subtle, but

overlooking their contributions typically leads to error bars that are both biased and not optimal. It

was recently found that the departure from a purely Gaussian analysis in a BAO measurement is hardly

detectable in a sample of 5000 N-body simulations (Takahashi et al., 2011). This measurement was

performed in an ideal environment, in which the estimate the power spectrum in also optimal and

unbiased way, with a survey geometry that assumes perfect seeing and periodic boundary conditions.

When navigating away from this special environment, the impact of non-Gaussian features needs to be

revisited.

In the real world, observations are subject to many effects that complicate the simulation picture:

presence of a survey selection function, redshift distortions, cosmic variance, shot noise, halo bias, etc.

1http://pan-starrs.ifa.hawaii.edu/2https://www.darkenergysurvey.org/


Understanding these effects and their combined dynamics is a major task that eventually needs to be

overcome. as these could become one of the main source of systematics uncertainty with the quality and

statistics of the next generation surveys.

For instance, data analyses that follow the prescriptions of Feldman et al. (1994) (FKP) or of Vogeley

& Szalay (1996)(PKL) assume that the underlying matter field is Gaussian when extracting the power

spectrum. Such a non-optimal measurement of the power spectrum has been shown to enhance the

non-Gaussian features, and the error bars on the BAO differ from Gaussian analyses by up to ten per

cent (Ngan et al., 2012). We therefore need to improve the way that data are analyzed, and the goal of

this series of paper is to explore how we could expand on the current methods and construct unbiased

non-Gaussian estimators of the matter power spectrum.

A major aspect common to non-Gaussian analyses is that they require an accurate measurement

of the full power spectrum covariance matrix, which is noisy by nature. Numerical convergence must

therefore be assessed, as an accurate measurement of N2 matrix elements requires much more then N

realizations. It is arguable that N2 independent measurements could be enough, depending on the final

measurement to be carried. Early numerical calculations were performed on 400 realizations (Rimes &

Hamilton, 2005), while Takahashi et al. (2009) have performed as many as 5000 full N-body simulations.

Ngan et al. (2012) has opted for fewer realizations, but complemented the measurements with noise

reduction techniques. In this context, it is often unclear how many simulations are required to reach

convergence to a well understood precision.

The first paper of this series, (Harnois-Deraps & Pen, 2012) (hereafter HDP1) addresses the issue of

bias caused by the presence of a selection function, and describes a procedure to estimate, for the first

time, non-Gaussian error bars on the power spectrum of galaxy surveys3. All deviations from Gaussian

calculations are parameterized by simple fitting functions, whose best fitting parameters are found from

sampling simulated dark matter particles. However, it does not address the question of halo bias nor of

shot noise, which surely will affect the best-fitting parameters (HDP3 in prep.)

In addition, this method assumes that the statistical features observed in N-body simulations are

unbiased representations of those from the actual Universe. This kind of correspondence is in fact

assumed in any external error analyses that are based on mock catalogues, either constructed from N-

body simulations or from semi-analytical techniques such as log-normal transforms of Gaussian density

fields (Coles & Jones, 1991) or PThalos (Scoccimarro & Sheth, 2002). Apart from the computational

3These error bars are unbiased, but unfortunately not optimal. As a matter of fact, the method still suffers from oneof the inconvenience of the FKP formalism, namely that the convolution with the survey selection function effectivelycorrelates the error bars. Removing this effect could be done by combining the methodology of HDP1 and that of PKL,but such a task is beyond the scope of this paper.


challenge, these external estimates may or may not have the same non-Gaussian properties and cosmology

as real data, and, to find out, precise ab initio simulations are unfortunately not available for real galaxy

surveys. Moreover, these semi-analytical techniques suffer from an additional problem: the accuracy at

which these can model the power spectrum covariance matrix is rather uncertain. The FKP and PKL

prescriptions are advantageous in that they provide internal estimates of the error bars, hence avoid

many possible biases that might contaminate external estimates.

In this second paper, we are asking the following two questions: 1- how can we possibly measure these

non-Gaussian features internally from a galaxy survey? and 2- how accurate are our measurements?

We first examine the problem of measuring error bars using a small number of measurements, with

minimal external assumptions. In this extremely low statistic environment, it has been shown that the

shrinkage estimation method (Pope & Szapudi, 2007) can minimize the variance between a measured

covariance matrix and a target, or reference matrix. Also important to mention is the attempt by

Hamilton et al. (2006) to improve the numerical convergence of small samples by bootstrap resampling

and re-weighting sub-volumes of each realizations, an approach that was unfortunately mined down by

beat coupling (Rimes & Hamilton, 2006). With the parameterization proposed in HDP1, our approach

has the advantage to provide more physical insights on the non-Gaussian dynamics. We also show that

the basic symmetries of the four-point function of the density field found in HDP1 allow us to increase

the number of internal independent subsamples by a large amount, with only a few quantifiable and

generic Bayesian priors.

In particular, it was shown that the contributions to the covariance matrix come in two parts: the

Gaussian zero-lag point, plus a broad, smooth, non-local, non-Gaussian component. The power spectrum

covariance C(k, k′, θ) is thus not isotropic, and only special subsets of the squeezed four point function

contribute to the non-Gaussian error bar. In this picture, both the Gaussian and non-Gaussian contri-

butions to the covariance matrix can be accurately estimated because many independent configurations

contribute, even within a single density field. However, any attempt to estimate them simultaneously,

i.e. without differentiating their different singular nature, results in large sample variance. It is exactly

this variance that forced previous measurements to be estimated from of a large number of simulations.

This paper takes advantage of this reduced variance to optimize the measurement of the covariance

matrix from only four N-body realizations. In such low statistics, the noise becomes dominant over a

large dynamical range, hence we further propose and compare a number of noise reduction techniques.

In order to gain physical insights on that noise, we paid special attention to provide error bars on each of

the non-Gaussian parameters and on the Fisher information. In this process, we compare and calibrate

our results with a larger sample of 200 realizations.


The final step of this paper describes a recipe to extract the non-Gaussian features of the power

spectrum covariance in the presence of a survey selection function, and in a low statistics environment,

thus completely merging the techniques developed here with those of HDP1. This takes us significantly

closer to a stage where we can perform non-Gaussian analyses of actual data, however redshift space

distortions (RSD), a key ingredient, is not included so far. To address this with the current approach

is much more resource demanding. The problem could be broken into a Legendre expansion: P (k, µ) =

P0(k)+P2(k)P2(µ)+P4(k)Pµ+ ... where Pi(k) are the multipoles, and Pℓ(µ) the Legendre polynomials.

One would then need to expand on the methodology of HDP1 to include an extra angle µ = cosθ in all

of the calculations, a serious numerical convergence challenge. A complete analysis would finally need to

include the auto- and cross-correlations between each of these first three multipoles. In addition Other

road maps might be more effective at addressing the RSD in the context of non-Gaussian analyses, and

for the moment, we focus our attention on the real space part only.

We first review in section 4.3 the formalism and the theoretical background relevant for estimates of

the matter power spectrum and its covariance matrix, exploiting the dual nature and symmetries of the

four point function. We then explain how to improve the measurement of C(k, k′) in section 4.4 with

three noise-reduction techniques, and compare the results with a straightforward bootstrap resampling.

In section 4.5 we measure the Fisher information of the noise filtered covariance matrices, and compare

our results against the large sample. We then discuss how to perform this parameter extraction in the

presence of a survey selection function in section 4.6, and conclude afterwards.

4.3 Extracting and filtering C(k,k’): bootstrap approach

In an ideal situation such as that which prevails in N-body simulations – i.e. periodic boundary conditions

and absence of survey selection function – the matter power spectrum P (k) is obtained by computing

the Fourier transform of a density contrast, which is defined as δ(x) ≡ ρ(x)/ρ − 1, where ρ(x) is the

density field and ρ its mean. Namely,

〈δ(k)δ∗(k′)〉 = (2π)3P (k)δD(k− k′) (4.1)

The Dirac delta function enforces the two scales to be identical, and the bracket corresponds to a volume

average. Simulated dark matter particles are assigned onto a grid following an interpolation scheme (CIC

in our case) (Hockney & Eastwood, 1981), and the volume average is performed over many independent

realizations. We refer the reader to HDP1 for details on the simulation suite, and briefly mention here

that each of the N = 200 realization is obtained by evolving 2563 particles with CUBEP3M (Merz et al.,


10−1

100

102

103

104

P(k

)[M

pc/

h]3

N=200N=4CAMB

10−1

100

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

∆P

/P

k[h/Mpc]

Figure 4.1: (top:) Power spectrum, estimated from 200 (thin line) and 4 (thick line) N-body simulations. In the N = 4

sample, the estimate on this sampling variance is poorly determined, as expected from such low statistics. The error bars

are the sample variance. (bottom:) Fractional error with respect to the non-linear predictions of CAMB (Lewis et al.,

2000).

2005) down to a redshift of z = 0.5, with cosmological parameters consistent with the WMAP+BAO+SN

five years data release. The isotropic power spectrum P (k) is finally obtained by taking the average over

the solid angle. Since this paper is addressing the issue of noise in the context of extremely low statistics,

we construct a small sample by randomly selecting N = 4 realizations among them; we hereafter refer

to these two samples as the N = 200 and the N = 4 samples. Fig. 4.1 shows the power spectrum as

measured in these two simulation samples, with a comparison to the non-linear predictions of CAMB

(Lewis et al., 2000). The simulations show a ∼ 10 per cent positive bias compared to the predictions,

a known systematic effect of the N-body code that happens when the simulator is started at very early

redshifts. Starting later would remove this bias, but then the covariance matrix, which is the actual

focus of the current paper, becomes less accurate. This is an unfortunate trade-off that will be avoided

in a future production run with the advent of a non-Gaussian initial condition generator.

The complete description of the uncertainty on P (k) must be estimated by constructing a covariance

matrix, defined as :

C(k, k′) = 〈∆P (k)∆P (k′)〉 (4.2)

where ∆P (k) refers to the fluctuations of the power spectrum about the mean. As mentioned in the

introduction, this matrix has been measured from a wide range of number of simulations, and little is


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4.2: (left:) Power spectrum covariance matrix, estimated from the N = 200 sample of N-body simulations. As

first measured by Rimes & Hamilton (2005), there is a large region where the scales are correlated by 60-70 per cent.

(right:) Power spectrum covariance matrix, estimated from only 4 simulations. As expected from such a low number of

measurements, the structure of the matrix is masked by a large white noise, especially in the low k-modes, which are

extracted from fewer grid elements.

known about the corresponding numerical convergence, since the error bars are rarely provided.

We present in Fig. 4.2 the covariance matrix, normalized to unity on the diagonal, and estimated

from the N = 200 and N = 4 samples. While the former is overall smooth, we see that the latter is, in

comparison, very noisy, especially in the large scales (low-k) regions. This is not surprising, since these

large scales are measured from fewer elements, hence intrinsically have a larger sample variance. This

illustrates why extracting the complete power spectrum covariance matrix from the data is a challenging

task: Fig. 4.2 seems too noisy to allow accurate non-Gaussian cosmology analyses.

This is nevertheless what we attempt to do in this paper. It was shown in HDP1 that there is an

alternative way to estimate this matrix, which first requires a measurement of C(k, k′, θ), where θ is the

angle between the pair of Fourier modes. We summarize here the properties of this four-point function:

• The four-point function receives a contribution from two parts: the degenerate singular configura-

tion with all k vectors equal – the zero-lag point – and the smooth non-Gaussian component.

• The zero-lag point corresponds to the Gaussian contribution, and needs to be treated separately

from the other points for k = k′.

• As the angle approaches zero, C(k, k′, θ) increases significantly, especially in the non-linear regime.


• In the linear regime, most of the contribution comes from the zero-lag point.

As discussed in section 5.3 and in the Appendix of HDP1, C(k, k′) can be obtained from the angular

integration of C(k, k′, θ):

C(k, k′) = dµC(k, k′, 0)δkk′ +∑

θ 6=0

C(k, k′, µ)dµ

≡ G(k, k′) +NG(k, k′) (4.3)

where C(k, k′, 0) is the zero-lag point, and µ = cosθ. It was shown that the difference between this

method and that of [Eq. 4.2] is at the per cent level, at least in the trans-linear and non-linear regimes4.

4.3.1 Diagonal elements

This break down of the covariance matrix is enlightening for many reasons, since we can ask ourselves

which of the two terms is the easiest to measure, which is the noisier, and how can we deal with the

noise in these two terms. The first term on the right hand side, G(k, k′), corresponds at the per cent

level – at least in the trans-linear and non-linear regimes – to the Gaussian contribution 2P 2(k)/N(k),

where N(k) is the number of modes in the k-bins. This comparison is shown in Fig. 4.3, where the error

bars are estimated from bootstrap resampling. In the N = 200 case, we randomly pick 200 realizations

500 times, and calculate the standard deviation across the measurements. In the N = 4 case, we select

4 realizations out of the 200, and repeat again 500 times. In the lower k regime, the simulations seem

to underestimate the Gaussian predictions by up to 40 per cent. This results seems surprising since

it occurs in the linear regime, which is the well described by Gaussian statistics; it indicates that the

G+NG decomposition is not exact there. This is not too surprising since for larger k-modes, the grid

discretization effect is amplified, and the angle between grid cell are less accurate. We therefore conclude

that our estimate of dµ is not very accurate in the linear regime, and that results involving a substitution

G(k, k′) → Cg should be interpreted with care at low-k.

We next look at the interplay between both G and NG on the diagonal of the covariance matrix.

We know from Fig.4.3 that the Gaussian term is well measured and has a rapid convergence, and that

the most optimal way to combine the error from the two terms is with a noise weighted sum. For the

sake of clarity, however, let us first consider the simplest case, where the error bars are obtained from

bootstrap resampling. We will come back to the noise weighting afterwards. We present in the left panel

of Fig. 4.4 the diagonal components of the covariance matrix, divided by the Gaussian prediction. The

4The agreement in the linear regime is reduced by discretization effects that cause the angles to be poorly determined,but this is exactly where the theory is well understood. The overall strategy thus puts priors on the non-Gaussian features,requiring that these vanish for scales large enough.


0.5

0.6

0.7

0.8

0.9

1

1.1

G200/C

g

10−1

100

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

k[h/Mpc]

G4/C

g

Figure 4.3: Comparison of the Gaussian term with the analytical expression. The top panel shows the ratio for N = 200,

while the bottom panel shows the ratio for N = 4. Error bars are from bootstrap resampling.

linear regime agrees well with the Gaussian statistics, and we observe strong deviations as the scales

become smaller. The shape of the ratio is similar in both the large and small sample, which lead us to

the conclusion that both can be parameterize the same way.

Following HDP1, we express the diagonal part of NG(k, k′) and C(k, k′) as5:

NG(k, k) =2P 2(k)

N(k)

(k

k0

)α

and C(k, k) =2P 2(k)

N(k)

(1 +

(k

k0

)α)(4.4)

In this parameterization, k0 informs us about the scale at which non-Gaussian departure become impor-

tant, and α is related to the strength of the departure as we progress deeper in the non-linear regime.

The best fitting parameters are presented in Table 4.1. A first important result is that the two sets of

parameters are consistent within 1σ, which tells us that the N = 4 sample has enough information to

extract non-Gaussian estimates of the diagonal elements of the covariance matrix. The fractional error

on both parameters is of the order of a few per cent in the large sample, and grows by about an order

of magnitude in the small sample. A second important results is that the fractional error on α is about

twice smaller than that on k0, which means that α is the easiest non-Gaussian parameter to extract.

We now ask which of G(k, k′) or NG(k, k′) has the largest contribution to the error on C(k, k). We

present in the right panel of Fig. 4.4 the fractional error on both terms, in both samples. We scale the

5The notation is slightly different than in HDP1, which expressed C(k, k) =2P2(k)N(k)

(

1 +

(

kα

)β)

≡2P2(k)N(k)

V (k). Note

the correspondance (α, β) → (k0, α).


Table 4.1: Best-fitting parameters used for the diagonal component of the covariance matrix, in our two samples.

Non-Gaussian features on the diagonal are parameterized as G(k, k) =2P2(k)N(k)

(

kk0

)α

. The error bars were obtained by

bootstrap resampling the measurements.

k0 α

N = 200 0.24± 0.01 2.24± 0.06

N = 4 0.20± 0.09 2.11± 0.54

bootstrap error by√N to show the sampling error on individual measurements, and observe that in

both cases, the non-Gaussian term dominates the error budget by more than an order of magnitude. As

expected, the non-Gaussian terms are noisier, and now we have at least a quantitative estimate of how

the noise compares to the Gaussian term.

4.3.2 Off-diagonal elements

The off-diagonal part of the covariance matrix, whose sole contribution comes from the non-Gaussian

term, is the noisiest of all. This can be attributed to the fact that there is a power of two more elements

to measure, from the same data size. Fig. 4.5 shows the ratio between the N = 4 and the N = 200

covariance matrices, where one scale is held fixed at k = 0.628hMpc−1. We observe that the largest

scales disagree by more than 100 per cent, largely due to the dominance of noise in the small sample.

The noise filtering technique we describe here has the most significant impact on the final result, and

consists in an Eigenvector decomposition of the cross-correlation coefficient matrix of C(k, k′). As first

discussed in Ngan et al. (2012), this method improves the accuracy of both the covariance matrix and

of its inverse. HDP1 further provides a fitting function for the Eigenvector, whose parameters can be

found even in a low statistics environment.

This decomposition is an iterative process that factorizes the cross-correlation coefficient matrix into

a purely diagonal component and a symmetric off-diagonal part. The latter is further decomposed into

a small set of Eigenvectors Uλ(k), and we keep only that which corresponds to the largest Eigenvalue

λ, which we denote U(k). This effectively puts a prior on the shape of the covariance matrix, but is

accurate at the few percent level, as shown in HDP1. The Eigenvector is presented in Fig. 4.6 for both


100

101

102

C200/C

g

Eq. 2Eq. 3GaussFit

10−1

100

10−1

100

101

102

k[h/Mpc]

C4/C

g

10−1

100

10−1

100

101

k[h/Mpc]

frac

err

GNG

Figure 4.4: (left) Ratio of the diagonal of the covariance matrix (from Eq. 4.3) with the Gaussian analytical expression.

The top panel shows the ratio for N = 200, while the bottom panel shows the ratio for N = 4. The error bars are from

bootstrap resampling. The thick solid lines are the fit functions, the dashed lines are obtained from [Eq. 4.2], and the

horizontal lines are the linear Gaussian predictions. (right) Fractional error on G and NG. The thin (thick) symbols

correspond to the N = 200 (N = 4) sample, and the error is from bootstrap resampling.

10−1

100

−1

−0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

k[h/Mpc]

C4/C

200

-1

Figure 4.5: Fractional error between the off-diagonal elements of the covariance matrices of the two samples. One scale

is held at k = 0.628hMpc−1. The error bars are from bootstrap resampling.


Table 4.2: Best fitting parameters for the Eigenvector in our two samples, parameterized as U(k) = A

(

k1

k+1

)−1

. The

error bars were obtained from bootstrap resampling and refitting.

λ A k1

N = 200 63.3± 0.7 0.129± 0.002 0.073± 0.011

N = 4 71.1± 5.0 0.119± 0.003 0.0066± 0.0185

samples, against a fitting function of the form6:

U(k) =A

k1/k + 1(4.5)

In this parameterization, A represent the overall strength of the non-Gaussian features, and k1 informs

us of the scale where cross-correlation becomes significant. The best fitting parameters are shown in

Table 4.2. We observe that the error bar on A is at the per cent level, and that the mean fluctuates

by only ten per cent between both samples. On the other hand, k1 is much harder to constraint: in

the N = 200 sample, the fractional error bar is about 15 per cent, and in the N = 4, its error bar is

larger than its mean. In other words, k1 is consistent with zero within 1σ in the small sample, which

corresponds to a constant Eigenvector. The noise is thus mostly concentrated there, and improving the

measurement on this parameter is one of the main tasks of this paper.

With the parameter fit to the Eigenvector, the N = 4 covariance matrix is no longer singular, as

shown in Fig. 4.7. It does exhibit a stronger correlation at the largest scales, compared to the matrix

estimated from the large sample. This difference roots in the fact that the Eigenvector remains high at

the largest scales, whereas the N = 200 drops. There is thus an overestimate of the amount of correlation

across the largest scales, which somehow biases the results on the high side. However, in the calculation

of Fisher information, these large scales are given such a low weight – they contain a small number of

independent Fourier modes – that their contribution is next to negligible. This figure shows that we do

recover the region where the cross-correlation coefficient is 60-70 per cent, also seen in the left panel of

Fig. 4.2. It is thus a significant step forward in accuracy compared to the Gaussian estimates.

6This parameterization of U(k) is equivalent to that of HDP1, but significantly simplified: we factorize γ outside of theparenthesis, and we set δ = 1. This choice is motivated by the fact that in the bootstrap estimate, the error bar on δ wasat the fraction of a per cent, indicating a weak dependence of the fit on this variable. Also, the Eigenvalue λ could beabsorbed into the proportionality constant A, but we keep it a separated parameter for reasons explained in section 4.4


0 0.5 1 1.5 2 2.50.02

0.04

0.06

0.08

0.1

0.12

0.14

U(k)

0 0.5 1 1.5 2 2.5

0.05

0.1

0.15

0.2

k[h/Mpc]

U(k)

Figure 4.6: Principal Eigenvector of the cross-correlation coefficient matrix. The top panel shows the N = 200, while

the bottom panel shows the N = 4 vector. The error bars are from bootstrap, and the thick solid lines are the fit obtained

with the parameters of Table 4.2.

10−1

100

10−1

100

k[h/Mpc]

k[h

/M

pc]

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4.7: Power spectrum covariance matrix, estimated from only 4 simulations, from the fit to the principal Eigen-

vector. There is a positive bias in the large scales (lower-left region) compared to the large sample (see left panel of Fig.

4.2), which is caused by a poor estimate of the Eigenvector for the lowest k-modes. Fortunately, this region has very little

impact on the Fisher information since it contains very few Fourier modes.


4.4 Optimal Estimation of C(k, k′): Noise Weighting

In the calculations of last section, we show that even in low statistics, we can extract non-Gaussian

parameters with the help of noise filtering techniques that assume a minimal number of priors on the

non-Gaussian features. As mentioned, all the error bars are obtained from bootstrap resampling, which

is only efficient in the large sample limit. In the N = 4 sample, which emulates an actual galaxy survey,

the large variance across the observation fields undermines the significance of our results, especially on

one of the parameter that describes the Eigenvector, k1. In this section, we expose two procedures that

optimizes the estimate of both the mean and the error on C(k, k′), and quantify the improvements on

all four non-Gaussian parameters (α, A, k0 and k1).

4.4.1 Noise-weighting NG(k,k’)

Before going any further, let us recall that in the bootstrap estimate of the error on C(k, k′), we resample

the measured C(k, k′, θ), integrate directly over the angle, and add the uncertainty of different angles

– including the zero-lag point – in quadrature. We propose here a different approach, which is based

on our knowledge that the distribution of NG(k, k′, θ) is much stronger for smaller angles. It is thus

natural that the mean and error from regions where the signal is cleaner should be weighted differently

that noisy regions. Our strategy is, first, to replace the quadrature sum of the error by a noise-weighted

sum in the angular integration of NG(k, k′, θ). This weighting strategy is also applied when including

G(k), which has the lowest noise.

To perform the noise-weighted angular integration, we first normalize each realization Ci(k, k′, θ) by

the mean of the distribution:

Di(k, k′, θ)± σD ≡ Ci(k, k

′, θ)

C(k, k′, θ)± σCC(k, k′, θ)

(4.6)

where σC is the standard deviation in the sampling of C(k, k′, θ) at each angle. As seen in Fig. 4.8,

the individual distributions of Di(k, k′, θ) are relatively flat, with a slight tilt towards smaller angles.

The two panels in this figure correspond to scales k = k′ = 2.1hMpc−1 and k = k′ = 0.31hMpc−1

respectively. In addition, the error bars σD get significantly smaller towards θ = 0 (or µ = 1). It is thus

a good approximation to replace each fluctuation by its noise-weighted mean over all angle Di(k, k′):

Di(k, k′) = σ2

T

∑

θ

Di(k, k′, θ)

σ2D

with σ−2T =

∑

θ

σ2D (4.7)

Doing so, the measurement of a matrix element Ci(k, k′) from a given realization is obtained by combining


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1

0

1

2

3

4

5

6

7

8

µ

D(k

i,k

j,µ)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−10

0

10

20

30

40

µ

D(k

i,k

j,µ)

Figure 4.8: (left:) Fluctuations in the angular dependence of the matter power spectrum covariance, for scales k =

k′ = 2.36hMpc−1, which correspond to scales of 2.7h−1Mpc. The error bars are the 1-σ standard deviation over our

200 samples. The dotted lines are the 200 realizations, while the thin solid lines represent the N = 4 sample. (right:)

Fluctuations for scales k = k′ = 0.314hMpc−1, which correspond to scales of 20.0h−1Mpc. The variations in the individual

fluctuations are much stronger than in the non-linear regime, a result that approaches the expected behaviour of Gaussian

random fields, where different measurements are less correlated.

the Gaussian and non-Gaussian contributions:

Ci(k, k′) = Gi(k, k

′) +∑

θ 6=0

Ci(k, k′, θ)w(θ)

= Gi(k, k′) +

∑

θ 6=0

Di(k, k′, θ)C(k, k′, θ)w(θ)

= Gi(k, k′) + Di(k, k

′)∑

θ 6=0

C(k, k′, θ)w(θ)

(4.8)

The Gi(k, k′) contribution is also added with a noise weighted sum and it that process, the uncertainty

on G is taken to be the sampling error. Note that in the above expressions, σC,D depend on the variables

(k, k′, θ), while σT depends on (k, k′). We have chosen not to write these dependencies explicitly in our

equations to alleviate the notation. The mean value of C(k, k′) computed with this method is identical

to the bootstrap approach ([Eq. 4.3]), since the realization average of Di(k, k′) is equal to unity by

construction. However, the intrinsic low noise of G, combined with the weighted error on NG, has a

beneficial effect on the error about C.

We show in Fig. 4.9 a comparison between the bootstrap sampling error bars on the C(k, k′) matrix,


10−1

100

100

105

∆C

0[M

pc/

h]6

10−1

100

10−3

10−2

10−1

100

k[h/Mpc]

∆C

0/C

0

Noise WeightedBootstrap

10−1

100

100

102

104

106

∆C

0[M

pc/

h]6

10−1

100

10−3

10−2

10−1

100

k[h/Mpc]

∆C

0/C

0

Noise WeightedBootstrap

Figure 4.9: (top left :) Comparison between bootstrap and noise weighted estimates of the sampling uncertainty on

the diagonal of the covariance matrix. We see that the improvement on the precision of this measurement is enhanced

significantly in the second scheme, even when very few samples are available. The thick lines are obtained from 200

realizations, the thin lines from only 4. (bottom left:) Fractional uncertainty comparison. (top right :) Same comparison,

for the case where one of the scales has been fixed to k = 0.628hMpc−1. (bottom right :) Fractional uncertainty comparison.

and the novel noise-weighted scheme. In the first case, we hold k′ = k, whereas in the second case,

we keep k = 0.628hMpc. The error bars achieved in the noise-weighted scheme are up to two orders

of magnitude smaller than the bootstrap errors, and the estimate of the error from the small sample is

already accurate. We also observe that the bootstrap fractional error is scale invariant, whereas that

from the noise-weighted method drops roughly as k−2 and thereby yields much tighter constraints on the

measured matrix elements. This comes from the fact that as we go to larger k-modes, the signal becomes

stronger, which improves the weighting. In both samples, by the time we have reach k = 0.1hMpc−1,

the improvement is about an order of magnitude, and at k = 1.0hMpc−1, the improvement is almost

two orders of magnitude. With a fractional error that small, our claim is that even with a handful of

realizations, the covariance matrix is precisely measured. It thus makes sense to use the noise filtering

techniques described in the last section to get around sampling variance and improve our estimate of

the true covariance matrix.


4.4.2 Wiener filtering

Since we now have accurate estimates of the error on the covariance C(k, k′) in both the N = 4 and

N = 200 samples, plus an accurate measurement of the mean in the large sample, we can attempt

to combine these three quantities in order to improve the mean of the small sample. The technique

described here is a Wiener filtering approach that uses noise properties in order to extract a signal closer

to a template.

In this case, the templates are the measurements estimated from 200 simulations, whose mean and

noise is well understood both for G and NG. The error on the former is obtained from bootstrap

resampling the zero-lag point, while the error on NG comes from the noise-weighted approach discussed

in section 4.4.1. Note that the Wiener filter could also come solely from bootstrap estimates of both G

and NG. We apply the Wiener filter on both quantities separately, and combine the results afterward.

Namely, we define our Wiener filter estimate as

CWF = GWF +NGWF (4.9)

with

XWF = X200 + (X4 −X200)

(σ2200

σ2200 − σ2

4

)(4.10)

where X = (G,NG). We present in Fig.4.10 the diagonal of the resulting covariance matrix, compared

to the N = 200 and N = 4 samples. We observe that the noise has decreased by almost a factor

of two compared to the original N = 4 sample. There is however a positive bias that appears for

k > 1.0hMpc−1, hence results beyond that point should be excluded from the analyses, or interpreted

carefully. The Wiener filtered cross-correlation matrix shown in Fig. 4.11 also shows that a large portion

of the noise has been filtered out. There are still undesired features in the linear regime, but the scales

with k > 0.5hMpc−1 are overall smooth and brought closer to the N = 200 sample. The actual accuracy

of this new matrix is discussed in a later section of this paper, when extracting Fisher information. Fig.

4.12 shows the effect on the Eigenvector of the cross-correlation matrix constructed with CWF . Most of

the benefits are seen for k < 1.0, which is the range we targeted to start with. For higher modes, the

Eigenvector drops, which is caused by the increased in the diagonal elements of CWF seen in Fig.4.10.

4.4.3 Noise weighted Eigenvectors decomposition

In this section, we describe a last noise filtering technique that utilizes our knowledge of the Eigenvectors,

Eigenvalues and their associated noise from the large sample to improve the way we perform the Eigen-


10−1

100

−0.5

0

0.5

1

1.5

k[h/Mpc]

∆C

/C

200

N=200N=4 (Naive)Wiener filter

Figure 4.10: Comparison between the diagonal elements of the covariance matrix with and without the Wiener filtering

technique. Results are expressed as fractional error with respect to the N = 200 sample.

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

k[h/Mpc]

k[h

/M

pc]

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4.11: Cross-correlation matrix after the Wiener filter has been applied.


decomposition of the small sample. It is a general strategy that is still under development, therefore we

do not show results in this paper.

The error on each Eigenvalue can be obtained by bootstrap resampling the 200 realizations, and

measuring the standard deviation about each λ. We know, for instance, that in the N = 200 sample,

only the first Eigenvalue really counts towards the signal; the others are more than an order of magnitude

smaller, and generally more noisy, as seen in Table 4.3. Since Eigenvector decompositions are independent

of matrix rotations, our strategy is rotate the N = 4 cross-correlation into a state where it is brought

closer to the N = 200 one, to apply a signal to noise weight to each elements and to rotate back. We

apply the following algorithm:

1. Rotate the noisy cross-correlation coefficient matrix ρ4 in the Eigenstates T of the large sample

equivalent ρ200

2. Weight the elements by the signal to noise ratio of the corresponding Eigenvalues

3. Perform an Eigenvector decomposition on the resulting matrix

4. Wiener filter the Eigenvalues

5. Undo the weighting

6. Rotate back

The rotation is defined as:

R = T−1ρT (4.11)

which reduces to the diagonal of Eigenvalues by construction in the case where ρ = ρ200. The weighting

is performed in two steps: 1) we scale each matrix element Rij by 1/√λiλj , which takes the R200 matrix

into the identity matrix, and 2) we weight the result by the signal to noise ratio of each λ. Combining,

we can write

Dij ≡ Rij

1√λiλj

(λiλjσiσj

)2

≡ Rijwiwj (4.12)

The resulting matrix D is then decomposed into Eigenvectors S,

D = SλDS−1 (4.13)

We bootstrap resample the realizations, measure S and N as the mean and noise on λD, and apply a

Wiener filter on these Eigenvalues with the replacement λD → λD

(S

S+N

). These Eigenvectors are then


Table 4.3: Ten largest Eigenvalues of the C(k, k′) decomposition, extracted from the angle averaged P (k) (following

equation 4.2) with the 200 realizations. The error bars are from bootstrap resampling the power spectrum measurements.

The Eigenvalues are organized in decreasing order.

63.3± 2.5 1.5± 2.1 0.4± 1.3 0.3± 0.1 0.3± 0.1

0.20± 0.08 0.19± 0.07 0.11± 0.05 0.08± 0.04 0.07± 0.03

0 0.5 1 1.5 2 2.5

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

k[h/Mpc]

U(k

)

N=200N=4Wiener Filtered

G(k) = 2P2(k)/N(k)

Figure 4.12: Effect of the different noise reduction techniques on the main Eigenvector of the power spectrum covariance

matrix. The parameters corresponding to each fit (not shown) are summarized in Table 4.4.

weighted back, we absorb the weights into the vectors as

Sij = Sij/wi (4.14)

and rotate back into the original space:

ρ4 = TλD ˜T−1 with T = TS (4.15)

If no weighting, this operation essentially does nothing, as the equivalence is exact: every rotation and

weights that are applied are removed. However, the Wiener filtering uses the structure of the noise to

improve the measurement of the Eigenvector.


Table 4.4: Best fitting parameters to the C/Cg ratio and to the Eigenvectors obtained from the different noise filtering

techniques.

α k0 A k1

Wiener filtered 0.191 2.011 0.1217 0.0232

G(k) = 2P (k)/N(k) 0.178 2.125 0.121 0.0184

4.5 Impact on Fisher Information

In analyses based on measurements of the matter power spectrum, the uncertainty on P (k) typically

propagates to cosmological parameters with Fisher matrices formalism (Tegmark, 1997). The Fisher

information, defined as

I(kmax) =∑

k,k′<kmax

C−1norm(k, k′)|k,k′<kmax

(4.16)

effectively counts the effective number density of degrees of freedom in a measurement. In the above

expression, Cnorm is simply given by C(k, k′)/(P (k)P (k′)), and reduces to 1/N(k) in the Gaussian case.

I(k) is an important intermediate calculation to the full Fisher matrix as it can be simply related to

theoretical calculations. For instance, it was first shown by Rimes & Hamilton (2005) that the number

of degrees of freedom increases in the linear regime, following closely the Gaussian prescription, but then

reaches a trans-linear plateau, followed by a second increase at even smaller scales. This plateau was

later interpreted as a transition between the two-haloes and the one-halo term (Neyrinck et al., 2006).

Non-linear estimates from the data have never been attempted so far, largely due to the fact that a

covariance matrix measured from a handful of observation fields is close to singular (as seen in Fig. 4.2).

With the noise reduction techniques described in this paper, the final matrix is no longer singular,

such that the inversion is finally possible. We present in the top panel of Fig. 4.13 the Fisher information

from the original matrix, for the matrix obtained after Eigen decomposition and for the one obtained

with the fitting functions, in the case where N = 200. For the N = 4 fields, the Eigen decomposed one

and the original curves quickly diverge. We therefore plot only the Fisher information obtained from

the fitting functions. We provide error bars for all of these curves, obtained from bootstrap resampling.

The Wiener filter described in section 4.4.2 enhances significantly the signal, and is also over plotted.

The bottom panel represents the fractional error between the different curves and the Eigen decomposed

N = 200 curve, which is our most accurate estimate of the underlying Fisher information.

The agreement between the Eigenvector decomposition and original curve in the N = 200 sample is

at the few percent level for k < 1.0hMpc−1. For the N = 4 sample, we recover a Fisher information


content that underestimates the large sample by less than 20 per cent for k < 0.3hMpc−1, and then

move on top and overestimate the large sample by up to 60 per cent away for k < 1.0hMpc−1. With

the Wiener filter, the maximal deviation is now reduced to less than 50 per cent, and replacing the

zero-lag point by the analytical prediction reduces the maximal deviation to 20 per cent. This represent

a significant step forward since estimates of I(k) on such low statistics finally possible. The deviations

seen in the N = 4 samples can be traced back to the fact that in the linear regime, the covariance matrix

exhibited a large noise that we could not completely remove. This extra correlation translates into fewer

degrees of freedom in the linear regime, which in turns shows up as a lower Fisher information content.

Better noise reduction techniques that specialize in the large scales could thus outperform the current

information measurement.

4.6 In the Presence of Surveys Selection Functions

Whereas results from the previous sections were based exclusively on simulations, this section now

discusses a toy example that illustrates how the best-fitting parameters can be extracted in the presence

of a survey selection functionW (x). The first paper of this series, HDP1, described FKP-like calculation

that incorporate an underlying non-Gaussian covariance matrix C(k,k′), which is known a priori, An

‘observed’ covariance matrix Cobs(k,k′) is related to the underlying one via a convolution:

Cobs(k,k′) ∝

∑

k′′,k′′′

C(k′′,k′′′)∣∣W (k− k′′)

∣∣2∣∣W (k′ − k′′′)∣∣2 (4.17)

In HDP1, this covariance matrix was calculated from N-body simulations, hence it is still an external

estimate of the error. What needs to be done for internal estimates is to walk these steps backward:

we know the selection function, we observe a noisy covariance matrix from a handful of independent

observations, and, ideally, we would like to deconvolve this matrix – a challenging six-dimensional in-

tegral to invert – and find the best-fitting parameters of the underlying covariance matrix. This brute

force approach is not realistic for the same reasons that prevents the forward convolution to be done

straightforwardly.

There is a solution, however, which exploits the fact that many of the terms involved in the forward

convolution are linear. It is thus possible to perform a least square fit for some of the non-Gaussian

parameters, knowing Cobs andW (x). We start by expressing the “true” non-Gaussian covariance matrix

into its parameterized form, following [Eq. 51-54] of HDP17, where the fitting functions Vℓ(k) and Uλ,ℓ(k)

are described in Tables 1 and 2 respectively, and CG(k) = 2P (k)N(k) . The complete matrix depends on a

7In the following discussion, we refer substantially to sections 7.2, 8 and 8.1 of HDP1, and we try to avoid unnecessaryrepetitions of lengthy equations.


10−1

100

101

102

103

104

I(k

)

OriginalEigen N=200Fit N=4Wiener FilteredG(k) = 2P(k)/N(k)Gauss

10−1

100

−0.4

−0.2

0

0.2

0.4

0.6

∆I/I

k[h/Mpc]

Figure 4.13: (top:) Fisher information extracted from our unfiltered N = 200 sample (thin dashed line with grey

shades), from the Eigenvector decomposed matrix (solid points with error bars), from fitting the N = 4 sample, leaving all

parameters free (thick dashed line), after Wiener filtering the covariance matrix (thick dot-dashed line), and after replacing

G(k) → 2P (k)/N(k).In the N = 4 case, the bootstrap error bars are very large, since bootstrap estimates are not accurate

in such low statistics systems; we therefore do not plot them. (bottom:) Fractional error between each of the top panel

curves and the N = 200 Eigenvector decomposed measurements.


total of 6 parameters to characterize the diagonal components, plus 45 to characterize the off-diagonal

elements of the Cℓ=0,2,4 matrices. These 51 parameters could be found all at once by a direct least

square fit approach, however some of them have more importance than other, hence we should ensure

convergence on them first. In particular, it was shown in figure 22 of HDP1 that most of the non-Gaussian

deviations come from C0, hence we start by finding the parameters that describe this contributions alone.

To simplify the picture even more, we decompose the problem one step further and focus exclusively on

the diagonal component, i.e.

C0(k, k′, µ) ≡ CG(k)δkk′δµ1

(1 +

(k

α

)β )(4.18)

In this case, the convolution with the survey selection function is greatly simplified as C0(k, k′, µ) is

isotropic. Following [Eq. 56] of HDP1, we can thus write directly:

Cobs0 (k, k′) =

∑

k′′

CG(k′′)

(1 +

(k′′

α

)β )∣∣W (k′′ − k)∣∣2∣∣W (k′′ − k′)

∣∣2 (4.19)

For each (k, k′) pair, we can solve for α and β, hence the final step is to find the values that minimize

the error.

The next step is to include the off-diagonal terms of the ℓ = 0 multipole, in which case the full C0 is

modelled. The “true” covariance matrix is now parameterized as

C0(k, k′, µ) ≡ C0(k, k

′, µ) +H0(k, k′) (4.20)

where

H0(k, k′) =

(F0(k)F0(k

′)− F 20 (k)δ(k − k′)

)(4.21)

and

F0(k) = Uλ,0(k)√λV0(k)CG(k) (4.22)

It is no longer isotropic, hence the forward convolution significantly increases in complexity. First, we

need to precompute the integral over θ′′ and φ′′ of the selection function (the X(k, k′′) function in [Eq.

57] of HDP1, with w(θ′′, φ′′) = 1), and write

Cobs0 (k, k′) = Cobs

0 (k, k′) +∑

k′′,k′′

H0(k′′, k′′′)

⟨X(k, k′′)X(k′, k′′′)

⟩(4.23)

The angle brackets refer to an average over θ and φ. There are five new best-fitting parameters that need

to be found from H0, and we can use our previous results on α and β as initial values in our 7 parameter

search. The first term on the right hand side was computed above, the X functions only depend on


W (k), hence we are left with N2 equations to solve for this set of parameters with a non-linear least

square fit algorithm.

Including higher multipoles can be done with the same strategy, i.e. progressively finding new

parameters from least square fits, using the precedent results as priors for the higher dimensional search.

One should keep in mind that the convolution with C2 and C4 becomes much more involving, since

the number of distinct X functions increases, as seen in Table 4 of HDP1. In the end, all of the 51

parameters can be extracted out of N2 matrix elements, in which case we would have fully characterized

the non-Gaussian properties of the covariance matrix from the data only.

4.7 Conclusion

In order to constrain the cosmology in an optimal and unbiased way, it is essential to extract accurate

non-Gaussian error bars on the matter power spectrum. One of the main difficulty is to reconcile the

intrinsic non-linear dynamics that couple the Fourier modes, with the the presence of a survey selection

functions. As explained in the first paper of this series on non-Gaussian error bars, traditional techniques

such as the FKP prescription tend to underestimate significantly the error bars on the power spectrum,

due to the assumption that the underlying field is Gaussian. In addition, mock catalogs have not been

shown to model the non-linear dynamics at an accuracy level required for sub-percent error bars.

The first paper showed that the angle-dependent covariance matrix C(k, k′, θ) receives a large con-

tribution from the zero-lag term (θ = 0 with k = k′), and a small but non-negligible from a smooth

non-Gaussian term. In this paper, we further demonstrates how this term connects with the Gaussian

prediction, and, based on this decomposition, we develop an alternative way to measure the mean and

uncertainty on the covariance matrix in the power spectrum of matter fields.

We show that the uncertainty on C(k, k′) can be obtained by noise-weighting the angular contribu-

tions, which significantly outperforms the bootstrap methods. For instance, in a sample of 200 realiza-

tions, the fractional error bars on C(k, k′) are up to two orders of magnitude smaller than with bootstrap.

We find that even with only four independent fields of observations, it is possible to achieve convergence

on the error on C(k, k′) at a degree comparable to the bootstrap obtained from 200 realizations.

The estimate of C(k, k′) obtained from such low statistics is still very noisy by nature. We develop

a series of techniques that improve the signal extraction by exploiting known properties about both

the noise and the signal. We quantify the performance by comparing the Fisher information between

different techniques to the large N = 200 sample, and find that we can recover the signal, from only

four realization, within less then 20 per cent up to k < 0.5hMpc−1, and less than 60 per cent up to


k < 1.0hMpc−1. Applying a Wiener filter brings the maximum deviation down to 50 per cent, and

replacing the zero-lag term by the analytical Gaussian prediction brings it down to 20 per cent, up

to k < 1.0hMpc−1 By comparison, the Gaussian approximation deviates by more than two orders of

magnitude there.

In this process, we describe the non-Gaussian departures with a series of parameters (α, k0, k1 and

A), and provide error bars for each of them and for the Fisher information. Here is a summary of the

convergence properties of the covariance matrix:

• The covariance matrix can be decomposed into a Gaussian term G(k) and a non-Gaussian term

NG(k, k′).

• The diagonal component of NG(k, k′) is modelled as a simple power law. In the N = 4 sample,

the slope α can be measured with a signal to noise ratio of 3.9, whereas the amplitude parameter

k0 has a ratio of 2.2.

• The off-diagonal elements of NG(k, k′) are parameterized by fitting the principal Eigenvector of

the cross-correlation coefficient matrix with two other parameters. The amplitude A has a signal

to noise ratio of 39.7, and the turnaround scale k1 is measured with a ratio of 0.36, which is thus

to hardest to extract. In the large N = 200 sample, it is measured with a ratio of 6.6, which is

style rather small.

This is important, as we now better understand the parts of the non-Gaussian signals that are the

noisiest, and therefore focus our efforts accordingly.

The step forward is significant, since we can first quantify the accuracy of our Fisher information,

plus our techniques are meant to be applied for measurements of the non-Gaussian error bars internally

from galaxy survey data. We finally suggest a strategy to extract these parameters from surveys with

a general selection function. The approach is iterative in the sense that we first find the parameters

that contribute the most to the non-Gaussian features, and source our results into higher dimensional

searches.

As mentioned in the introduction, many challenges in our quest for optimal and unbiased non-

Gaussian error bars are not resolved yet, and it is the goal of this series of paper to pin down these issues

one by one. For instance, many results, including all of the fitting functions from HDP1, were obtained

from simulated particles, whereas actual observations are performed from galaxies. It is thus important

to repeat the analysis with simulated halos, in order to provide a non-Gaussian prescription consistent

with galaxy surveys. Also, we have not yet addressed the question of redshift distortions, which will


inevitably influence the accuracy of the fitting functions. In addition, simulations were performed under

a specific cosmology and at z = 0.5, and it is important to have some understanding of the cosmology

and redshift dependences of the non-Gaussian parameters. These dependences are minimal in internal

estimates, since the best fitting parameters are obtained from the survey itself.

Acknowledgments

The authors would like to thank Chris Blake for reading the manuscript and providing helpful comments

concerning the connections with analyses of galaxy surveys. UP also acknowledges the financial support

provided by the NSERC of Canada. The simulations and computations were performed on the Sunnyvale

cluster at CITA.

Chapter 5

Optimizing the Recovery of Fisher

Information in the Dark Matter

Power Spectrum

5.1 Summary

We combine two complimentary techniques – Wavelet Non-Linear Wiener Filter (WNLWF) and density

reconstruction – to quantify the recovery of Fisher information about the amplitude of the matter

power spectrum that is lost in the gravitational collapse. We compute a displacement fields, in analogy

with the Zel’dovich approximation, and apply a Wavelet Non-Linear Wiener Filter that decomposes

the reconstructed density fields into a Gaussian and a non-Gaussian component. From a series of 200

realizations ofN -body simulations, we compute the recovery performance for density fields obtained with

both dark matter particles and haloes. We find that the height of the Fisher information trans-linear

plateau is increased by an order of magnitude at k = 0.7hMpc−1 for particles, whereas either technique

alone offers an individual recovery boost of only a factor of three to five. We conclude that these

two techniques work in symbiosis, as their combined performance is much stronger than their individual

contribution. When applied to the halo catalogues, we find that the reconstruction has only a weak effect

on the recovery of Fisher Information, while the non-linear wavelet filter boosts the information by about

a factor of five. We also observe that non-Gaussian Poisson noise saturates the Fisher information, and

that shot noise subtracted measurements exhibit a milder information recovery.

120

Chapter 5. Optimizing the Recovery of Fisher Information 121

5.2 Introduction

Understanding the nature of dark energy has been identified internationally as one of the main goal of

modern cosmology (Albrecht et al., 2006), and many dedicated experiments attempt to constrain its

equation of state: LSST1 (LSST Science Collaborations et al., 2009), EUCLID2 (Beaulieu et al., 2010),

JDEM3 (Gehrels, 2010), CHIME4 (Peterson et al., 2006), SKA5 (Schilizzi, 2007; Dewdney et al., 2009),

BOSS6 (Schlegel et al., 2009a) and Pan-STARRS7. One of the favoured technique involves a detection of

the Baryonic Acoustic Oscillations (BAO) signal (Seo & Eisenstein, 2003, 2005; Eisenstein et al., 2006;

Seo & Eisenstein, 2007), which has successfully constrained the dark energy parameter in current galaxy

surveys (Eisenstein et al., 2005; Tegmark et al., 2006; Percival et al., 2007; Blake et al., 2011). The

analyses are based on a detection of the BAO wiggles in the matter power spectrum, which act as a

standard ruler and allow one to map the cosmic expansion.

With the new and upcoming generation of dark energy experiments, the precision at which we will

be able to measure the cosmological parameters is expected to drop at the sub-per cent level, therefore

it is essential to understand and suppress every sources of systematic uncertainty. In a BAO analysis,

one of the main challenge is to extract an optimal and unbiased observed power spectrum, along with its

uncertainty; the latter propagates directly on the dark energy parameters with Fisher matrices (Fisher,

1935; Tegmark et al., 1997). This task is difficult for a number of reasons.

For instance, the scales that are relevant for the analyses sit at the transition between the linear

and the non-linear regime, at least for the redshift at which current galaxy surveys are sensitive, hence

the underlying uncertainty on the matter power spectrum is affected by the non-linear dynamics. These

effectively couples the phases of different Fourier modes (Zhang et al., 2003) and the Gaussian description

of the density fields has been observed to fail (Meiksin & White, 1999; Rimes & Hamilton, 2005, 2006;

Neyrinck et al., 2006; Neyrinck & Szapudi, 2007). For an estimate of the BAO dilation scale to be

robust, one must therefore include in the analysis the full non-linear covariance of the power spectrum.

Although results from Takahashi et al. (2011) seem to suggest that non-Gaussianities had no real effect

on the final results, it was recently shown that this was only true if the original power spectrum was

measured in an unbiased and optimal way, which is rarely the case (Ngan et al., 2012). Otherwise, the

discrepancy on the constraining power is at the per cent level. One of the ways to reduce the impact

1http://www.lsst.org/lsst/2http://www.congrex.nl/09c08/3http://science.nasa.gov/missions/jdem/4http://www.physics.ubc.ca/chime/5http://www.skatelescope.org/6http://cosmology.lbl.gov/BOSS/7http://pan-starrs.ifa.hawaii.edu/public/


of the non-linear dynamics is to transform the observed field into something that is more linear. Over

the last few years, many “Gaussianization” techniques (Weinberg, 1992) have been developed, which

all attempt to undo the phase coupling between Fourier modes. The number of degrees of freedom –

i.e. uncoupled phases – can be simply quantified by the Fisher information, and recovering parts of this

erased information can lead to improvements by factors of a few on cosmological parameters.

For example, a density reconstruction algorithm (Eisenstein et al., 2007; Noh et al., 2009; Padman-

abhan et al., 2009) has been shown to reduce by a factor of two the constraints on the BAO dilation scale

(Eisenstein et al., 2007; Ngan et al., 2012). It effectively moves the simulated (or observed) objects to an

earlier time based on the Zel’dovich approximation, in a state where the density field is more Gaussian,

i.e. where departures from Gaussianity occur at smaller scales. This technique was recently applied on

the SDSS data (Padmanabhan et al., 2012) to improve the BAO detection, with small modifications

to the algorithm such as to correct for the survey selection function and redshift space distortions. As

discussed therein, an important issue is that two main mechanism are reducing our ability to measure

the BAO ring accurately: 1) a large coherent ∼ 50 Mpc infall of the galaxies on to overdensities, which

tends to widen the BAO peak, and 2) local non-linear effects, including non-linear noise, which also erase

the smallest BAO wiggles. Reconstruction addresses the first of these mechanisms, and it is important

to know whether something can be done about the second, after reconstruction has been applied.

Wavelet Non-linear Wiener Filters (hereafter WNLWF, or just wavelet filter) were used to decompose

dark matter density fields (Zhang et al., 2011) and weak gravitational lensing κ-fields (Yu et al., 2012)

into Gaussian and non-Gaussian parts, such as to condense in the latter most of the collapsed structure;

the Gaussian part was then shown to contain several times more Fisher information than the original

field. Other methods include log-normal transforms (Neyrinck et al., 2009; Seo et al., 2011), Cox-Box

(Joachimi et al., 2011), running N -body simulation backwards (Goldberg & Spergel, 2000), or direct

Gaussianization of the one-point probability function (Yu et al., 2011), just to name a few. This technique

seems perfectly suited to address the issue of non-Gaussian noise described above.

Our focus, in this paper, is to discuss how two of these techniques can be used in conjunction

to maximize the recovery of Fisher information about the amplitude of the matter power spectrum.

The cosmological application is immediate, as a higher information content in the range k ∼ 0.5 −

1.0hMpc−1 means smaller error bars on the BAO signal, hence tighter constraints on dark energy. Not

all combinations of Gaussianization techniques are advantageous, however. It was recently shown (Yu

et al., 2012) that WNLWF and log-transforms are not combining in an advantageous way. On one hand,

if the log-transform is applied onto a Gaussianized field, the prior on the density field is no longer valid,

and the log-transform maps the density into something even less Gaussian. On the other hand, it was


shown that the log-transform is less effective than WNLWF alone at recovering Fisher information, at

least on small scales. Applying the filter after the log-transform does not improve the situation, since the

Gaussian/non-Gaussian decomposition is less effective. In other words, the Fisher information that the

log-transform could not extract is not recovered by WNLWF, and we are better off with the WNLWF

alone.

It seems, however, that this unfortunate interaction is not a constant across all combinations. In this

paper, we discuss how non-linear Wiener filters, constructed in Wavelet space, can improve the results

of a density reconstruction algorithm, which takes the density back in time using linear perturbation

theory. Our first result is that these two techniques work well together, in the sense that the final Fisher

information recovery is larger than the two techniques stand alone. We first obtain these results with

particle catalogues extracted from N -body simulations, and extend our techniques to halo catalogues,

which provide a sampling of the underlying matter field that is much closer to actual observations.

The structure of the paper is as follows: in Section 5.3, we briefly review the theoretical background

of the density reconstruction and WNLWF, and review how we extract the density power spectra, their

covariance matrices, and the Fisher information. We discuss our results in Section 5.4 and conclude in

Section 5.5.

5.3 Theoretical Background

5.3.1 Numerical Simulations

Our sample of 200 N -body simulations are generated with CUBEP3M, an enhanced version of PMFAST

(Merz et al., 2005) that solves Poisson equation with sub-grid resolution, thanks to the p3m calculation.

Each run evolves 5123 particles on a 10243 grid, and is computed on a single IBM node of the Tightly

Coupled System on SciNet (Loken et al., 2010) with ΩM = 0.279, ΩΛ = 0.721, σ8 = 0.815, ns = 0.96

and h = 0.701. We assume a flat universe and find the initial position and velocity of the particles at

zi = 50 with Zel’dovich approximation. Each simulation has a side of 322.36h−1Mpc, and we output

the particle catalogue at z = 0.054. We search for haloes with a spherical over-density algorithm (Cole

& Lacey, 1996) executed at run time, which sorts the local grid density maxima in descending order

of peak height, then loops over the cells surrounding the peak centre and accumulates the mass until

the integrated density drops under the collapse threshold of 178, and finally empties the contributing

grid cells before continuing with the next candidate, ensuring that each particle contributes to a single

halo. Halo candidates must consist of at least one hundred particles, ensuring the haloes are large and


collapsed objects. The centre-of-mass of each halo is calculated and used as position, as opposed to its

peak location, even though both quantities differ by a small amount. We mention here that algorithms

of this kind have the unfortunate consequence to create an exclusion region around each halo candidate,

thus effectively reducing the resolution at which the halo distributions are reliable. Each field contains

about 88, 000 haloes, for a density of 2.6 × 10−3h3Mpc−3. For comparison, this is about eight times

larger than the BOSS density (Schlegel et al., 2009a).

5.3.2 Density reconstruction algorithm

We use a density reconstruction algorithm that is based on the linear theory prediction, first found by

Zel’dovich (Zel’Dovich, 1970), that couples the density field δ(q, t0) to the displacement field s(q) via

δ(q, t0) = −∇ · s(q) (5.1)

In the above expression, q is the grid, or Lagrangian, coordinate, and the displacement field is obtained

in Fourier space as

s(k) = − ikk2δ(k, t0)F (k) (5.2)

where F (k) = exp[−(kR)2/2] is a smoothing function suppressing features smaller than R = 10h−1Mpc.

Such calculations are commonly used for the generation of initial conditions in N -body simulations, and

are accurate as long as the smallest scales probed are still in the linear regime at the starting redshift.

In the case where the particles – or haloes – to be displaced are not at t0, one must subtract from the

result the displacement field from the grid location (see Noh et al. (2009) for a detailed explanation of

this technique). In our numerical calculations, the particles are assigned on to a 10243 density grid with

a cloud-in-cell interpolation scheme (Hockney & Eastwood, 1988), and the displacement fields is actually

obtained by finite-differentiation of the potential field.

5.3.3 Wavelet non-linear Wiener filter (WNLWF)

In this subsection we briefly review the WNLWF algorithm, and direct the reader to Zhang et al. (2011);

Yu et al. (2012) for more details.

We consider in this paper the Daubechies-4 (Daubechies, 1992) discrete wavelet transform (DWT),

which contains certain families of scaling functions φ and difference functions (or wavelet functions) ψ.

The density fields are expanded into combinations of these orthogonal bases, and weighted by scaling

function coefficients (SFCs) and wavelet function coefficients (WFCs). In our WNLWF algorithm, we


deal only with the latter, each of which characterizes the amplitude of the perturbation on a certain

wavelength and at a certain locations.

In the three dimensional case, the properties of each perturbation depend on three scale indices

(j1, j2, j3) – controlling the stretching of the wavelet Daubechies-4 functions – and three location indices

(l1, l2, l3) – controlling their translation. Specifically, in a given dimension, the grid scale corresponding

to a specified dilation is L/2j (L = 1024 in our case), and the spatial location is determined by lL/2j <

x < (l + 1)L/2j. After the wavelet transform, all SFCs and WFCs are stored in a 3-dimensional field,

preserving the grid resolution (see Fang & Thews (1998); Press et al. (1992) for more details).

Our non-linear Wiener filter (NLWF) construction strategy relies on the fact that in wavelet space,

the non-Gaussianities are clearly characterized in the probability distribution function (PDF) of the

WFCs ǫj1,j2,j3;l1,l2,l3 . We thus construct our filter by splitting the wavelet transform of the original

density, which we label D, into a Gaussian (G) and a non-Gaussian (N) contribution. Since wavelet

transforms are linear operations, this Gaussian/non-Gaussian decomposition also happens in real space

when we wavelet transform back the contributions. We can thus write in wavelet space and real space

respectively :

D = G+N and d = dG + dNG (5.3)

where the original density (d) is expressed as the sum over a Gaussian contribution (dG) and a non-

Gaussianized contribution (dNG). Our goal is to design a filter that concentrates most of the collapsed

structures in dNG, and therefore produces dG that are closer to linear theory.

The NLWF acts on individual wavelet modes, which are defined as combinations (not permutations)

of all WFCs having the same three scale indices (j1, j2, j3). For each wavelet mode, the NLWF is

determined completely by the PDF of the corresponding WFCs, fPDF (x), which is constructed by

looping over the other three indices (l1, l2, l3)8

We then fit this PDF with the analytical function presented in Yu et al. (2012) (equation (15)):

fPDF(x) =1√

πs1−αs2

Γ(12αs2)

Γ(12αs2 − 1

2 )(s2 − x2)−

αs2

2 (5.4)

and extract the two parameters α and s. These are actually dependent on the second and the fourth

central moment of the PDF, m2 and m4, hence in practice, we measure the moments first, then extract

α and s via:

α =5m4 − 9m2

2

2m2m4and s =

√∣∣∣∣2m2m4

m4 − 3m22

∣∣∣∣. (5.5)

8We note that each wavelet mode has its own PDF, hence we should really be writing f l1,l2,l3PDF (x), but we omit the

location indices to clarify the notation.


Here α− 1

2 = σG is the standard deviation of the Gaussian (central) PDF, describing the variation of the

Gaussian contribution of the fluctuations in this wavelet mode.

We construct the Gaussian and non-Gaussian filter functions, as functions of x, for this wavelet mode:

wG(x) = −σ2G(ln f)

′(x)

x=

(1 +

x2

s2

)−1

, (5.6)

wNG(x) = 1 +σ2G(ln f)

′(x)

x= 1−

(1 +

x2

s2

)−1

, (5.7)

apply them to all spatial indices (l1, l2, l3) of this wavelet mode and decompose each WFC into two

components – Gaussian and non-Gaussian.

We emphasize on the fact that the filter functions depend only on the parameter s, which is the

full width at half maximum (FWHM) of the Gaussian NLWF window function wG. It characterizes

the extent of the departure from a Gaussian PDF: the greater the s, the smaller the departure from

Gaussian statistics. The same decomposition is performed on the reconstructed density fields and on

those obtained from the halo catalogues. In this paper, we do not make use of the information contained

in the non-Gaussian component and simply discard it, although it serves as a powerful probe of small

scale structures and could help identifying haloes in a (Gaussian) noisy environment.

5.3.4 Information recovery

The calculation of uncertainty about dark energy cosmological parameters is based on a propagation of

the uncertainty about the matter power spectrum. In this process, the number of degrees of freedom

– i.e. the Fisher information – contained in the field is directly related to the constraining power. In

this section, we review how the Fisher information about the amplitude of the matter power spectrum

is calculated from simulated dark matter particles and haloes.

The power spectrum P (k) of a density contrast δ(x) is calculated in a standard way:

P (k) = 〈P (k)〉 = 〈|δ(k)|2〉 (5.8)

where the angle brackets refer to an average over our 200 different realizations and over the solid angle.

The uncertainty about the power spectrum is estimated from a covariance matrix C is defined as

C(k, k′) ≡ 1

N − 1

N∑

i=1

[Pi(k)− 〈P (k)〉][Pi(k′)− 〈P (k′)〉], (5.9)

where N is the number of realizations and 〈P (k)〉 is the mean angular power spectrum over all realiza-

tions. The cross-correlation coefficient matrix is somehow more convenient to plot since it has higher


contrasts, and is defined as:

ρ(k, k′) =C(k, k′)√

C(k, k)C(k′, k′)(5.10)

The diagonal is normalized to one, and each element represents the degree of correlation between the

scales (k, k′). In the Gaussian approximation, the density is completely characterized by the power

spectrum. Namely,

CG(k, k′) =

2P 2(k)

N(k)δkk′ (5.11)

Consequently, ρG is identical to the identity matrix. As expected from the theory of structure formation,

the non-linear collapse of the matter density field tends to couple the Fourier modes, which are otherwise

independent, starting from the smallest scales and progressing towards larger scales with time. This

coupling is responsible for highly correlated region in ρ, and can be understood in terms of higher-order

corrections, including the bispectrum, the trispectrum, etc.

The Fisher information measures the number of independent Fourier modes in a density field up

to a resolution scale kmax. We see from equation (5.11) that dividing the covariance by the (square of

the) power spectrum is proportional to the number of independent measurements N(k). Therefore, the

normalized covariance is defined as:

Cnorm(k, k′) =

C(k, k′)

P (k)P (k′), (5.12)

Then, the number of degrees of freedom up to a scale kmax is obtained by inverting the corresponding

sub-sample of the normalized matrix, then summing over all the elements:

I(kmax) =

kmax∑

k,k′

C−1norm(k, k

′). (5.13)

The inversion of the covariance matrix involved in the calculation of the Fisher information amplifies

the noise, hence such measurements typically requires a very strong convergence on the forward matrix.

This can otherwise lead to biases of a few per cent on important derived quantities like the stretching

of the BAO scale (Ngan et al., 2012). Generally, a forward matrix that is closer to diagonal contains

more Fisher information; the theoretical maximum corresponds to the Gaussian case, where all modes

are independent. Any non-vanishing off-diagonal element reduces the information. In addition, in the

presence of noise, the inverse of an unbiased estimator is not an unbiased estimator of the inverse (Hartlap

et al., 2007). In the current paper, we used 200 simulations to extract the power spectrum covariance

matrix about 15 k−bins, therefore the estimate of inverse we present would be about 9 per cent biased

on the high side, at least in the linear regime. However, the unbiased estimator proposed by Hartlap


et al. (2007) is only valid in Gaussian statistics, and our measurements are sitting at the transition

between the linear and the non-linear regime, hence it is not clear how the proposed correction would

actually improve the estimate. In any case, this inversion bias is constant for all the measurements we

made, and since we are ultimately interested about the ratio between them, the correction factor would

cancel out. For these reasons, we do not include this correction factor in the figures.

5.4 Results

In this section, we describe and quantify our ability at recovering Fisher information with our two

Gaussianization techniques, for density fields measured with simulated particles and haloes. We recall

that the later is much closer to actual observations, since galaxies trace highly collapsed structures.

5.4.1 Density fields

To illustrate the effect of different Gaussianization techniques, we present in Fig. 5.1 the projections

through a thickness of 50 cells of a given realization after density reconstruction alone, after WNLWF

alone, and with both techniques applied. We observe that the reconstruction reduces the size of each

halo, as expected from this algorithm: particles attempt to travel out of the gravitational potential.

WNLWF has a slightly different visual effect on the density: it removes most of the smallest structure

perturbations, leaving behind the larger ones. As discussed in Pen (1999), the geometry of the Cartesian

wavelet sometimes leaves behind a grid patterns, which only affects the smallest scales of the power

spectrum – in this case k > 8.0h/Mpc – and has no impact on the scales we are interested in. The

combination of both techniques is presented in the middle right panel, and visually presents the least

amount of collapsed structures. We also show the non-Gaussian part of the wavelet filter with and

without reconstruction in the bottom panels. The largest peaks and sharpest structures are indeed

filtered out.

Fig. 5.2 shows the power spectrum of the dark matter particles and haloes before and after the

Gaussianization techniques. We first observe that the measurement from the original particle field

agrees at the few per cent level with the non-linear predictions obtained from CAMB (Lewis et al.,

2000; Smith et al., 2003) up to k ∼ 2.0hMpc−1, which sets the resolution scale of our power spectrum

measurements. As expected from WNLWF, the Gaussian component of WNLWF preserves the power

on linear scales – up to k ∼ 0.1hMpc−1 – while signals from trans-linear and non-linear scales are mostly

transferred to the non-Gaussian contribution of the WNLWF decomposition, which explains the drop

in power. It is exactly for this reason that the shot noise, which typically contributes only at relatively


Dis

tanc

e [h

−1 M

pc]

0 50 100 150 200 250 3000

50

100

150

200

250

300

0 50 100 150 200 250 3000

50

100

150

200

250

300

0

5

10

15

20

25

30

Dis

tanc

e [h

−1 M

pc]

0 50 100 150 200 250 3000

50

100

150

200

250

300

0 50 100 150 200 250 3000

50

100

150

200

250

300

0

5

10

15

20

25

30

Dis

tanc

e [h

−1 M

pc]

Distance [h−1Mpc]

0 50 100 150 200 250 3000

50

100

150

200

250

300

Distance [h−1Mpc]

0 50 100 150 200 250 3000

50

100

150

200

250

300

0

5

10

15

20

25

30

Figure 5.1: Projections through a thickness of 50 cells of one of the realizations. In each panel, the side is 322.36h−1Mpc,

and the image contains 10242 pixels. Top left is the original field, top right is the field after linear density reconstruction,

middle left is the wavelet filtered field (Gaussian part), middle right is the result of wavelet filtering the reconstructed

field. The non-Gaussian part of the wavelet filtered fields are shown in the bottom panels with (right) and without (left)

density reconstruction. To ease the visual comparison, each panel shows the same overdensity range and saturates for

denser regions, i.e. all pixels with δ > 30 are black.


small scales, is filtered out by the WNLWF. At the same time, we note that the wavelet filtered power

spectrum actually traces quite well the linear predictions, to within a factor of two, at all scales.

The density reconstruction algorithm also has a significant impact on the shape of the power spec-

trum, since particles are pumped out of the gravitational potential. As a result, power from k >

0.1hMpc−1 is transferred to larger scales, as seen in the figure. It is not entirely surprising that this

scale also corresponds to that where linear and non-linear calculations depart from one another: both

techniques are affecting the statistics of the fields in the trans-linear regime to increase the information

about the power spectrum’s amplitude contained therein. When looking at the halo measurements, we

observe that the original and reconstructed power spectra are dominated by shot noise at scales smaller

than k ∼ 1.0hMpc−1. For reasons explained above, this noise is strongly suppressed by the wavelet

filter.

In practice, a common way to deal with Poisson noise is to compute the cross power spectrum

between two populations randomly selected out of the original catalogue. The shot noise is averaged out

in this operation, and the signals left behind are stronger on large and intermediate scales. Small scales

are typically anti-correlated due to the ‘halo exclusion’ effect, which is a result of our halo-finder: the

algorithm effectively collapses all the structure of a given halo to a single point, and leaves empty the

region surrounding the centre-of-mass. In that regime, then, the cross-power spectrum becomes negative,

and should therefore be excluded from the analyses. Although precise and robust, this procedure is hard

to apply to all the cases under study in this paper, since the density reconstruction algorithm requires

an accurate measurement of the gravitational potential. The halo exclusion effect already undermines

the extracted gravitational potential, the technique would loose all its accuracy if we removed half of

the haloes. To estimate the noise-free power spectrum, we use another common approach that consists

in subtracting from the measurement a flat shot noise, defined as Pshot = L3/Nhalos. This technique is

much faster and completely compatible with the requirements of our algorithms, however, the resulting

noise appears to be overestimated. A better estimates of the noise in the halo distribution is probably

the product of this Pshot times a scale dependent bias, but in order to avoid any systematics associated

with such a bias, we decided to preserve the original, conservative noise estimate.

To quantify how this simple noise filtering performs, we show in Fig. 5.3 a comparison between the

original halo power spectrum from the full catalogue, the cross spectrum, and the shot-noise subtracted

power. We observe that the two shot noise subtraction techniques agree up to k = 0.3hMpc−1, beyond

which the P (k) − Pshot(k) approach filters out more power in comparison; by k = 1.0hMpc−1, the left

over signal is smaller by a factor of 2.4. For even smaller scales, the shot noise starts to dominates

the signal. This difference between the two shot noise subtraction techniques means that our approach


is not optimal, as we are loosing too much power at small scales; the results on scales in the range

[0.3− 1.0]hMpc−1 are therefore not as optimal as one would wish.

5.4.2 Covariance matrices

The two Gaussianization techniques that are discussed in this paper both attempt to bring cosmological

information, or degrees of freedom, back to the power spectrum. Consequently, the covariance matrices

of the Gaussianized fields are expected to be more diagonal. The top left panel of Fig. 5.4 shows the

cross-correlation coefficient matrix of the original particle fields in the upper triangle, and the wavelet

filtered ones on the lower triangle. To ease the comparison between the figures, we show in the panel the

positive components only, and present in the insets the negative entries. There is a mild anti-correlation

(less than 10 per cent) in some matrix elements of the original fields, which comes from residual noise in

the largest scale. This noisy effect that has very little impact, except for the undesired featured that the

information content is allowed to exceed that of the Gaussian case. To correct for it, we take advantage

of the fact that the underlying matrix, when estimated from thousands of simulations, has no negative

elements (Takahashi et al., 2009; Ngan et al., 2012). We therefore assume that the negative elements we

are measuring are noisy, and get rid of them by replacing the matrix by its absolute value. This surely

introduces a bias since it is a one way levelling, i.e. no positive elements are lowered. As a result, the

inverse of the matrix, which we are interested in, is probably a little bit on the low side. However, this

only occurs at k ∼ 0.15hMpc−1, and the bias becomes negligible as high k-modes are included. The

top right panel shows, on the upper triangle, the results after the density reconstruction, then, on the

lower triangle, the measurements after both technique have been executed. The off-diagonal elements

of the covariance matrix are reduced by 20-40 per cent by both Gaussianization techniques individually,

whereas the combined techniques suppress the elements by 40-70 per cent, compared to the original.

The bottom panels of Fig. 5.4 show the same measurements, when carried on halo fields. The wavelet

filter produces a band of negative elements, correlating the k > 1.0hMpc−1 – Poisson dominated – with

all scales. This anti-correlation does not carry any physics about the signal in it, hence these scales should

be left out or carefully interpreted in future analyses. Most off-diagonal elements are about 30 per cent

less correlated than in the unfiltered matrix, showing that wavelet filtering is also very efficient on halo

fields. We also observe that the density reconstruction algorithm has very little impact on the correlation

of the halo measurements. This is caused by the fact that haloes are non-overlapping by construction,

hence the region of exclusion prevents an accurate construction of the gravitational potential9. This is a

9We could of course improve the performance of the reconstruction technique by using the gravitational potentialmeasured from simulated particles, which we have at hand. Even in a data set, it is in principle possible to combine


10−1

100

101

102

103

104

105

P(k

)[(M

pc/

h)3

]

Original (particles)ReconstructedWavelet filteredbothCAMBcyclic (haloes)

10−1

100

101

−1

−0.5

0

0.5

1

k[h/Mpc]

∆P

/P

Figure 5.2: (top:) Power spectra of the original and Gaussianized fields, from simulated particles (symbols + solid lines)

and haloes (symbols + dashed line, in red on the on-line version). The linear and non-linear predictions from CAMB are

shown by the thick solid lines, and the Poisson noise corresponding to the halo population is shown with the thin dotted

line. We observe that the wavelet filtered particles densities trace the linear CAMB predictions, at most within a factor of

two. (bottom:) Fractional error between the curves of the top panel and the non-linear prediction from CAMB. We observe

that the particle power spectrum deviates by more than 10 per cent for k > 2.0hMpc−1, which sets the resolution limit of

our simulations. This scale is represented by the vertical line in both panels. We observe that in linear regime, the wavelet

filter preserves the agreement with the predictions, whereas the reconstruction tends to increase the power spectrum by

about 20 per cent. This is not a surprise since one effect of the algorithm is to transfer power from small to large scales. In

the non-linear regime, however, the power spectrum is highly suppressed by the wavelet filtering process, which factorizes

the structures into the non-Gaussian contribution. We measure a linear bias of about 1.2 in all halo measurements. The

original and reconstructed halo power spectra are shot noise dominated at scales smaller than k ∼ 1.0hMpc−1, a scale

that is strongly suppressed by the wavelet filter.


10−1

100

100

101

102

103

104

k[h/Mpc]

P(k

)[h/M

pc]

3

pphalocross−powerP(k) − P

shot

Pshot

Figure 5.3: Cross power spectra of a single density field, constructed from a random separation of the haloes onto two

distinct fields, whose Fourier transform are combined. The dashed line is the halo power spectrum of the full population,

the dotted line is the power spectrum of the particles, the straight line is the Poisson noise estimate, the open symbols

represent the cross spectrum of the two randomly selected populations, and the thick solid line is the shot noise subtracted

power.

strong limit of the technique, as real galaxy data behave much more like halos than particles. However,

data allow for higher multiplicity in the galaxy population of haloes, which to improves the construction

of the gravitational potential, hence we can expect a better gain in observations. We also mention here

that the scale at which the halo exclusion effect occurs is rather deep in the non-linear regime, hence any

structure that is miss-modelled by our halo-finder would have fallen in the the non-Gaussian contribution

anyway. For this reason, the Gaussian component of the wavelet filtered fields does not suffer from this

systematic effect. When we combine both technique, most of the diagonalization comes from wavelet

filter. This can be seen visually by comparing the lower triangles of both bottom panels.

5.4.3 Fisher information

When we extract the Fisher information from the covariance matrices presented above, we expect the

original particle fields to exhibit the global shape first measured in Rimes & Hamilton (2005). Namely,

the information should follow the Gaussian predictions on large scales, then reach a trans-linear plateau

where the gain is very mild as one increases the resolution of the survey, then hit a second rise on scales

smaller than about 1.0hMpc−1. We first see from Fig. 5.5 that we are able to recover those results, plus

those of Ngan et al. (2012), which showed that the density reconstruction algorithm can raise the height

independent measurements of the potential, obtained say with weak lensing tomography. This is an interesting avenuethat is, however, beyond the scope of this paper.


10−1

100

10−1

100

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−0.1

−0.05

0

10−1

100

10−1

100

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−0.2

−0.15

−0.1

−0.05

0

10−1

100

10−1

100

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−0.4

−0.3

−0.2

−0.1

0

10−1

100

10−1

100

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−0.2

−0.15

−0.1

−0.05

0

Figure 5.4: (top-left:) Cross-correlation coefficient matrix associated with the particle power spectra. The top triangle

represents measurements from the original matter fields, while the lower triangle elements are from fields after the NLWF

has been applied. The inset quantifies the amount of anti-correlation between the measurements. (top-right:) The top

triangle represents measurements from the reconstructed matter fields, while the lower triangle are from fields that are

first reconstructed, then wavelet filtered – still using particles as tracers. (bottom-left:) Cross-correlation coefficient matrix

associated with the power spectrum measurements from the simulated haloes. The top triangle represents measurements

from the original halo fields, while the lower triangle are from fields that are wavelet filtered. (bottom-right:) The top

triangle represents measurements from the reconstructed halo fields, while the lower triangle are from fields that are first

reconstructed, then wavelet filtered.


of the trans-linear plateau by a factor of a few. We also recover the results from Zhang et al. (2011)

and obtain a similar gain with the wavelet non-linear Wiener filtering technique. In this plot, we have

divided the Fisher information by the simulation volume, such as to represent the numbers density of

degrees of freedom. This choice makes it easier to compare results from other authors that were obtained

with different volumes.

As mentioned in the introduction, it was shown by Yu et al. (2012) that different Gaussianization

techniques do not always combine well. In the current case, however, we observe that on all scales,

the Fisher information from the combined techniques are larger than the sum of the two separate

contributions. For k > 0.6hMpc−1, notably, we are able to extract more than ten times the Fisher

information of the original particles fields, whereas individual techniques offer a recovery of about a

factor of four. This symbiosis effect grows larger as one goes to smaller scales.

When considering the halo fields, we observe in Fig. 5.6 that the density reconstruction technique,

taken alone, has little impact on the recovery of information, due to a poor modelling of the gravitational

potential. In contrast, wavelet filtering recovers five times more information by the time we have reached

k = 1.0hMpc−1, before shot noise subtraction. The Poisson noise is a non-Gaussian effect, which also

saturates the Fisher information. A hard limit one can think of is the following: the number of degrees of

freedom can not exceed the number of objects in our fields of view. We therefore plot the (non-Gaussian)

Poisson noise limit as a flat line corresponding to the halo number density, and observe that the original

halo Fisher information approach but never exceed that limit. Wavelet fields, however, reduces the

Poisson noise significantly, hence allows the information to reach higher values. We also see that shot

noise subtracted Fisher information curves show a lower information recovery, which means that the

number density needs to be high enough in order to maximize the recovery. In that case, however, only

the region for k = 0.7hMpc−1 can be trusted, since our simple noise subtraction technique becomes non-

reliable at smaller scales. As mentioned in section 5.3.1, the halo density is about eight times larger than

current spectroscopic surveys. Next generation experiments and current photometric redshift surveys

have a much larger number counts, hence the corresponding Poisson noise limit will be much higher.

5.5 Discussion and Conclusion

This paper explores the recovery of Fisher information about the amplitude of the matter power spectrum

with the combined use of two Gaussianization techniques – wavelet non-linear Wiener filtering and

density reconstruction – and show that we can extract an order of magnitude more information than the

original fields, and that the conjunction of both methods outperforms the performance of the individual


10−1

100

10−6

10−4

10−2

100

I(k

)[h/M

pc]

3

10−1

100

5

10

15

20

25

k[h/Mpc]

I(k

)/I o

(k)

Original (particles)ReconstructedWavelet filteredbothGauss

Figure 5.5: (top :) Cumulative information contained in the dark matter power spectra of the original and Gaussianized

fields from particles. As in Fig. 5.2, the dots represent the original fields, the open circles show the results after our density

reconstruction algorithm, the squares correspond to the wavelet filtered fields, and the stars represent a combination of

both techniques. The analytical Gaussian (i.e. linear) Fisher information curve is shown with the thick solid line. These

two Gaussianization techniques are shown to work in conjunction, such that on all scales, their combined effect recovers

the largest amount of information. For k > 0.6hMpc−1, the improvement on particles is more than an order of magnitude.

(bottom :) Ratio of the lines presented in the top panel with the original fields.


10−1

100

10−6

10−4

10−2

100

I(k

)[h/M

pc]

3

10−1

100

2

4

6

8

10

k[h/Mpc]

I(k

)/I o

(k)

Original (haloes)ReconstructedWavelet filteredbothGausscyclic (shot noise subtracted)

Figure 5.6: (top :) Cumulative information contained in the dark matter power spectra of the original and Gaussianized

fields from haloes. The analytical Gaussian (i.e. linear) Fisher information is shown with the thick solid line. Reconstruction

offers only a mild improvement on the Fisher information when taken alone, whereas wavelet filter recovers three times

more information by k = 0.7hMpc−1. The flat line corresponds to the halo number density, and the dashed lines are

for shot noise subtracted calculations (in red on the on-line version). We observe that the (shot noise included) halo

information saturates at the number density, as expected from non-Gaussian Poisson density fields. Wavelet filtered halo

densities have a lower shot noise, hence can exceed this Poisson limit. (bottom :) Ratio of the lines presented in the top

panel with the original fields. We see that shot noise subtracted fields shows a milder information recovery. This due to

the presence of non-Gaussian Poisson noise in the original fields, which was largely removed by the wavelet filter, thus

boosting the performance. The arrow points towards the shot noise subtracted measurements that are robust.


techniques. We also reproduce the calculations on halo catalogues and find that 1) with the spherical over

density halo finder, the density reconstruction has only a mild impact on its own, due to an inadequate

modelling of the gravitational potential near the centre-of-mass, 2) wavelet filtering is very efficient and

recovers about five times more information by k = 0.7hMpc−1 compared to the original halo fields,

and 3) after applying a conservative shot noise subtraction technique, the combined methods recovers

about three times more Fisher information than in the original fields by k > 0.7hMpc−1. These results

suggest that non-Gaussian error estimates about the matter power spectrum can be minimized in the

trans-linear regime, a fact that directly affect the uncertainty about the BAO measurements. In other

words, optimizing the recovery of Fisher information with such techniques benefits the constraints about

cosmological parameters such as dark energy equation of state. We recall that our focus here is not

so much to improve the estimate of the power spectrum, but to minimize its uncertainty. Moreover,

we have used the Gaussian part of the Wiener filtered data, and it will be interesting to see how the

non-Gaussian component can be incorporated in the analyses, especially in the presence of a high level

of Gaussian noise. Interestingly, we find that in both particles and haloes, the recovery of information

from the combination of the two Gaussianization techniques is significantly larger than that from the

individual contributions, which suggests that these two techniques are somehow complimentary. In this

work, we have performed the reconstructed density before the wavelet filter, since the former technique

still exhibits a non-Gaussian component at small scales, which is caught by the latter. The inverse order

would not have performed that well, since the reconstruction technique needs and accurate measurement

of the gravitational potential about the collapsed structures, a dynamical regime that is highly modified

by the wavelet filtering.

Acknowledgements

This work was supported by the National Science Foundation of China (Grants No. 11173006), the

Ministry of Science and Technology National Basic Science program (project 973) under grant No.

2012CB821804, and the Fundamental Research Funds for the Central Universities. Computations were

performed on the TCS supercomputer at the SciNet HPC Consortium. SciNet is funded by: the Canada

Foundation for Innovation under the auspices of Compute Canada; the Government of Ontario; Ontario

Research Fund - Research Excellence; and the University of Toronto. UP and JHD would like to

acknowledge NSERC for their financial support.

Chapter 6

Gravitational Lensing Simulations :

Covariance Matrices and Halo

Catalogues

6.1 Summary

Gravitational lensing surveys have now become large and precise enough that the interpretation of the

lensing signal has to take into account an increasing number of theoretical limitations and observational

biases. Since the lensing signal is the strongest at small angular scales, only numerical simulations can

reproduce faithfully the non-linear dynamics and secondary effects at play. This work is the first of a

series in which all gravitational lensing corrections known so far will be implemented in the same set of

simulations, using realistic mock catalogues and non-Gaussian statistics. In this first paper, we present

the TCS simulation suite and compute basic statistics such as the second and third order convergence

and shear correlation functions. These simple tests set the range of validity of our simulations, which

are resolving most of the signals at the sub-arc minute level (or ℓ ∼ 104). We also compute the non-

Gaussian covariance matrix of several statistical estimators, including many that are used in the Canada

France Hawaii Telescope Lensing Survey (CFHTLenS). From the same realizations, we construct halo

catalogues, computing a series of properties that are required by most galaxy population algorithms.

139

Chapter 6. Gravitational Lensing Simulations 140

6.2 Introduction

The latest measurements of the cosmic microwave background (CMB) (Jarosik et al., 2011; Planck

Collaboration et al., 2011) and of large scale galaxy surveys (York et al., 2000; Colless et al., 2003;

Semboloni et al., 2006) suggest that the Universe is mostly filled with dark energy and dark matter.

In that so called concordance, or standard, model of cosmology, the matter that is actually observed

accounts for only five per cent of the total energy. Improving our knowledge about this dark sector

is one of the biggest challenge physicists and astrophysicists are facing, and it was soon recognized

that an international effort, which would combine complimentary techniques such as baryonic acoustic

oscillations, type 1A supernovae, weak lensing and cluster growth, could lead to tight constraints on

dark energy parameters (Albrecht et al., 2006).

Whereas the latest analyses (Percival et al., 2001; Eisenstein et al., 2005; Tegmark et al., 2006;

Percival et al., 2007; Benjamin et al., 2007; Komatsu et al., 2011) were able to achieve per cent level

precision on most parameters, next generation surveys, including LSST (LSST Science Collaborations

et al., 2009), Euclid1 (Laureijs et al., 2011), SKA2 (Lazio, 2008), Pan-STARRS3, VST-KiDS4, DES5

are designed to reach the sub-per cent level. To achieve such performance, any systematic or secondary

effect needs to be understood with at least the same level of accuracy.

In the context of global dark energy efforts, weak lensing analyses are particularly appreciated for

their ability to detect dark matter structures with a minimal amount of bias. They are based on the

measurement of the degree of deformation caused by the foreground matter structures, which act as a

lenses, on background light sources. The signal allows us to characterize the average mass profile of

foreground lenses, which typically consist of groups or clusters of galaxies of different type, morphology

and colour, generally centred on a dark matter halo. Although the shape of the 2-point cosmic shear

signal depends on many cosmological parameters, it is especially powerful at constraining a combination

of the normalization of the matter power spectrum σ8 and the matter density Ωm. The degeneracy

between σ8 and Ωm can then be broken with measurements of the skewness and other higher-order

statistics (Bernardeau et al., 1997). High precision measurements of these two parameters are relevant

for dark energy when combined with complimentary techniques – standard ruler with baryonic acoustic

oscillations, or redshift-luminosity distance with type 1A supernovae for instance – however it was

recently shown that weak lensing is also a standalone probe. The signal indeed has dependencies on

1http://www.euclid-ec.org2http://www.skatelescope.org3http://pan-starrs.ifa.hawaii.edu/4http://www.astro-wise.org/projects/KIDS/5https://www.darkenergysurvey.org/


the redshift-distance relation, the growth factor and the non-linear clustering (Huterer, 2002; Albrecht

et al., 2006; Hoekstra & Jain, 2008).

In order to match the statistical and systematic accuracy of upcoming surveys, it is thus of the utmost

importance to minimize all of the theoretical uncertainties associated with the weak lensing technique.

The accuracy at which one can model this signal depends on a number of things. First, one must assess

the reliability of the modelled lenses and sources distributions, which mainly depend on the accuracy of

the underlying matter density field. Next comes the calculation of the propagation of light, which can

be done with varying degrees of precision. Finally, the modelled signal depends on the accuracy of the

galaxy population algorithm and on our understanding of all secondary effects.

Early generations of analytical calculations were performed using linear theory (see Bartelmann &

Schneider, 2001, for a review), and used, for instance, Zel’dovich approximation to produce late redshift

lenses, which assumes a linear matter power spectrum. These are known to underestimate the amount

of structure over a large dynamical range. Indeed, it was shown that for sources at redshift z = 1, the

projection of a non-linear density field on to a light cone impacts the lensing signal at angles up to a

few tens of arc minutes (Jain et al., 2000). Better results were obtained from higher-order perturbation

theory (Bernardeau et al., 2002), within the halo model (Cooray & Sheth, 2002) or by using the non-

linear predictions of Smith et al. (2003). Unfortunately, these models fail at recovering accurate lensing

signals at small angles, largely due to inaccuracies in the non-linear calculations. Similarly, estimates

based on mock catalogs made with log-normal densities (Coles & Jones, 1991) are fast and convenient,

and were shown to yield results very similar to N-body simulations (Hilbert et al., 2011) in the trans-

linear regime. However, it was discovered at the same time that log-normal mocks tend to overestimate

the covariance for scales smaller than a few arc minutes, which are critical for our current work. For

optimal measurements, it was soon realized that one needs to rely on accurate N-body simulations for

modelling the non-linear density field (Blandford et al., 1991; Premadi et al., 1998; White & Hu, 2000;

Jain et al., 2000; Hilbert et al., 2009).

For the sake of constraining the dark matter and dark energy parameters, having an accurate the-

oretical model of the weak lensing signal is not enough. In addition, the description of the statistical

uncertainty associated with the measurements needs to be as accurate as the signal itself. Current data

analyses work under the assumption that a Gaussian description of the field is accurate enough, given the

other sources of uncertainty involved in the measurements. Although this approximation is reasonable

for existing surveys, it will no longer be adequate for the next generation of surveys, which will be much

more sensitive to the non-linear scales.

In particular, the non-linear nature of the density field on small scales tends to correlate measurements


in a different way. For instance, the Fourier modes of the density fields grow independently in the linear

regime, but couple together in the trans-linear regime (Coles & Chiang, 2000), giving rise to non-Gaussian

features in the three-dimensional matter power spectrum (Meiksin & White, 1999; Rimes & Hamilton,

2005; Neyrinck et al., 2006; Takahashi et al., 2009; Ngan et al., 2012; Harnois-Deraps & Pen, 2012).

These propagate in weak lensing measurements, as observed in simulations (Takada & Jain, 2009; Dore

et al., 2009; Sato et al., 2009, 2011a) and in the data (Lee & Pen, 2008). These correlations are in fact

decreasing the number of degrees of freedom contained in the probed volume, hence reduce the power

at which one can constrain cosmology (Dore et al., 2009; Lu et al., 2010). It is therefore essential to

measure accurately this effect at a resolution that matches that of modern surveys, i.e. at the sub-arc

minute level.

Two of the main limitations of existing simulation suites is that they are either not resolving small

enough scales, or they are limited in terms of number of realizations6. For example, the Coyote Universe

simulation suite models many cosmological volumes of 1300 h−1 Mpc per side, organized in three series

for each cosmology. The ‘L-series’ consists of 16 realizations and covers the lower-k modes only, while

the ‘H-series’ covers the quasi-linear regime. These two series are produced with a PM code, and their

combination can probe scales up to k ∼ 0.43hMpc−1. In addition, a single realization is ran with a

tree-PM code, and resolves the k ∼ 1 hMpc−1 scales (Lawrence et al., 2010). These allow for a very

accurate measurement of the mean power spectrum, but are not adequate to achieve convergence on

the covariance matrix. The analyses carried by Kiessling et al. (2011b) was based on the SUNGLASS

pipeline (Kiessling et al., 2011a), in which 100 simulations were produced with 5123 particles and a box

size of 512 h−1Mpc. For some measurements 100 realizations could be enough, but unfortunately small

scales are not resolved well enough, and the agreement with theoretical predictions deviates by more

than 10 per cent for ℓ > 2000. What we need is to push the resolution limits by at least an order of

magnitude, reaching ℓ ∼ 10000. Similarly, the 1000 realizations produced by Sato et al. (2009) have a

low resolution compared to our needs.

At this point, running a new series is necessary, and the requirements are a) sub arc-minute accuracy

on the lensing signal, and b) large statistics, for convergence of the covariance matrix. The first goal of

this paper is to describe a new set of simulations, the TCS simulation suite, that fulfils both of these

requirements, and to provide a robust description of the non-Gaussian uncertainty on the 2- and 3-point

estimators that are commonly used in weak lensing analyses. It is constructed from the CUBEP3M

6Generally speaking, a covariance matrix with N2 elements will have converged if estimated with much more than Nsimulations. Numerical convergence tests on these matrices have shown that N2 realizations yield an error of the order often per cent on each element (Takahashi et al., 2009; Dore et al., 2009).


N-body code, with eight times more particles than the above mentioned SUNGLASS series, in a volume

more than forty times smaller, thus probing much deeper in the non-linear regime. Our choice of N-body

code is largely motivated by the fact that in the absence of high density regions, P3M Poisson solvers

are much faster than tree-PM codes, and the particle-particle interactions at the sub-grid level enhance

the resolution significantly compared to PM codes.

With the forecasted accuracy of the next generation of surveys, many secondary effects, that were

previously overlooked or neglected, now need to be carefully examined, since they are likely to contribute

to a large portion of the theoretical uncertainty. We gather here the principal secondary signals.

• The impact of intrinsic alignment needs to be quantified in order to calibrate the lensing signal

(Heymans et al., 2004). This effect is caused by the fact that galaxies that live in a same cluster are

subject to a coherent tidal force, which tends to compress them along the direction to the centre

of mass of the system (Heavens & Peacock, 1988; Schneider & Bridle, 2010).

• Another secondary effect that needs to be examined is the so-called ‘intrinsic alignment-lensing

correlation’ (sometimes referred to as ‘shear-ellipticity correlation’ or contamination), a correlation

that exists between the intrinsic alignment of the foreground galaxies and the shear signal of the

same galaxies on background sources. For this effect to occur, the foreground system needs to be

relaxed enough such that the orientation of the foreground galaxy correlates with the tidal field it

is subjected to (Hirata & Seljak, 2004).

• Source clustering is another important secondary effect, which is caused by the fact that sources

are not uniformly distributed : regions of the sky with more sources are likely to provide a stronger

weak lensing signal (Bernardeau, 1998).

• Also to be tested is the possible intrinsic alignment of galaxies with voids, an effect which was

previously found to be consistent with zero (Heymans et al., 2006).

Some of these effects have already been studied in simulations (Heymans et al., 2006; Semboloni et al.,

2008, 2011b), but the statistical accuracy and the resolution were limited, such that these previous works

need to be extended. More importantly, the secondary effects have been studied separately so far, and

we do not yet understand how they blend together. This is the second goal of this paper: we set the stage

to start quantifying in details how these weak lensing secondary effects interact, with sub-arc minute

accuracy. Because the only way to quantify their impact in the data is by measuring their combined

contribution in mock galaxy catalogues, our long term plan is to construct a large sample of mock

galaxy catalogues to quantify both the mean and the uncertainty on these secondary effects. For this,


we need to test separately the accuracy of the underlying density fields, the halo finder algorithm, the

galaxy population scheme and the proposed weak lensing estimator (see Forero-Romero et al. (2007)

for example). This paper is addressing the first step in this construction, that is the determination of

key properties of the underlying dark matter haloes, in the cosmological context under study. Galaxy

populations, secondary effects and cosmology forecasts will be part of future papers. On the longer term,

we are hoping that our catalogues and gravitational lenses will be used to test new ideas that might

contribute to the systematics of weak lensing signals, or to other aspects of cosmology.

One of the limitations of our simulation suite is that it does not include baryonic matter, hence does

not model any of the baryonic physics that might influence the dark matter distribution. Recent work

suggests that effects such as AGN feedback and supernovae winds could impact noticeably the matter

distribution in the Universe (Semboloni et al., 2011a). This might be significant for the interpretation

of the lensing signal, especially at small angular scales, and one could imagine that future generations of

simulations could implement all the effects we are discussing here plus the effect of baryonic feedback.

This paper is organized a follow. In Section 6.3, we briefly review the theoretical background relevant

for weak lensing studies, then we describe in Section 6.4 our design strategy, our N-body simulations,

as well as our numerical methods to construct the lines-of-sight and the gravitational lenses. In Section

6.5, we quantify the accuracy of our simulations and of our lines-of-sight. We present the weak lensing

estimators and their non-Gaussian uncertainty in Sections 6.6, 6.7 and 6.8, and conclude in Section 6.9.

6.3 Theory of Weak Lensing

The propagation of a photon bundle emitted from a source located at an angle β and observed at θ in

the sky is characterized by a Jacobian matrix A(θ):

Aij(θ) =dβ

dθ= (δij −Ψij(θ)) (6.1)

where the matrix Ψij(θ) encapsulates the distortion of the two-dimensional image. At first order, it is

determined by three components, namely a convergence κ and two shear components γ1 and γ2 that

combine together into a complex shear γ ≡ γ1 + iγ2. At second order, an asymmetric factor ω appears

in the off-diagonal elements, but it has a negligible contribution in realistic situations (see, for example,

the Appendix of Schneider et al. (1998)), hence we drop it. We thus write

Ψ =

κ+ γ1 γ2

γ2 κ− γ1

(6.2)


All of these elements are locally determined by the Newtonian potential Φ via:

κ =Φ,11 +Φ,22

2, γ1 =

Φ,11 − Φ,22

2, γ2 = Φ,12 (6.3)

where ‘, i’ refers to a derivative with respect to the coordinate i. For a source located at a comoving

distance χs, the projected distortion is computed as:

Ψij =2

c2

∫ χs

0

Φ,ij

D(χ)D(χs − χ)

D(χs)dχ (6.4)

where c is the speed of light. The angular diameter distance D(χ) depends on the curvature:

D(χ) =

K(− 1

2) sinh(K

1

2χ) for K > 0

χ for K = 0

−K(−1

2) sin(−K 1

2χ) for K < 0

(6.5)

with

K =

(H0

c

)2

(1 − Ωm − ΩΛ) (6.6)

H0 is Hubble’s parameter, Ωm and ΩΛ are respectively the ratio of the mass and dark energy densities

to the critical density.

The convergence field is particularly interesting theoretically since it relates, through Poisson’s equa-

tion, to the matter density contrast δ:

2κ = ∇2Φ(x) =3

2ΩmH

20 (1 + z)δ(x) (6.7)

with

δ(x) =ρ(x)− ρ

ρ(6.8)

Following standard notation, ρ is the average matter density in the Universe and ρ(x) is the local density.

In this paper, we are assuming a flat Universe, in which D(χ) takes the simplest form. Substituting [Eq.

6.7] in [Eq. 6.4], we can extract the projected convergence κ up to a distance χs as:

κ(θ, χs) ≃∫ χs

0

W (χ)δ(χ, θ)dχ (6.9)

where W (χ) is defined as

W (χ) =3ΩmH

20

2c2(1 + z)g(χ) (6.10)

and

g(χ) = χ

∫ ∞

χ

dχ′n(χ′)

(χ′ − χ

χ′

)(6.11)


In this paper, we work under the single source plane approximation, both for illustrative purposes and

to disentangle cleanly the sources from the lenses. In this case, g(χ) reduces to χ(1 − χχs

) for a source

plane located at χs. In future papers, the source distribution will be made more realistic by constructing

n(χ) from observed surveys. Finally, once we have a convergence field, we extract the shear field by

solving for the gravitational potential from [Eq. 6.3] (Kaiser & Squires, 1993)7.

6.4 Numerical Methods

As mentioned in the introduction, gravity is a non-linear process, and the predictions from the linear

theory of large scale structures are only valid on the largest scales. In the context of weak lensing,

photons trajectories are probing a broad dynamical range, including galactic scale structures, where

the matter fields are highly non-linear. Although higher-order perturbation theory can describe such

systems, the accuracy of the calculations are limited by the complex dynamics. We therefore rely on

N-body simulations to generate accurate non-linear densities, through which we shoot photons rays and

extract the resulting distortion matrix. In this section, we describe some of the considerations one must

keep in mind when performing such calculations.

6.4.1 Constructing the lenses

N-body simulations need to be optimized according to the specific measurements to be performed. In the

current paper, we attempt to estimate the covariance matrices for a number of weak lensing estimators,

hence we need a large number of realizations. In addition, we are interested in resolving the sub-arc

minute signal, hence the simulation grid needs to be fine. Ideally, one would simulate the complete past

light cone that connects the observer to the light sources all at once. Unfortunately, for sources that

extend to redshift of a few, this is not possible since the far end of the cosmological volume is at an

earlier time than the close end. It is, however, the only way one could model the largest radial modes of

a survey. Luckily, it was realized that these modes contribute very little (Limber, 1953): the coherence

scales of the largest structures significant for the signal rarely extends over more than a few times the

size of large clusters. Simulation volumes of the order of a few hundreds of h−1Mpc’s per side are thus

generally adequate. These simulated boxes can then be stacked to create a line-of-sight (LOS), inside of

which photons are shot.

7For this operation, we are working under a flat sky approximation, which allows us to perform the Fourier transformsin the traditional plane wave basis, and to simplify the derivatives as a simple finite difference. Also, one must be carefulabout the method used to perform this calculation, since the Universe is not periodic, while simulations usually are. Theedge effects can therefore contaminate the calculations, hence it is necessary to somehow pad the boundaries.


One can use a different simulation for each redshift box, as done by (White & Hu, 2000), but this

method is CPU consuming, since a single LOS that extends to z 3 involves between 10 and 40 N-body

simulations. This is unrealistic for covariance matrix measurements, which require hundreds of these

high precisions LOS. We opted for the now common work around developed by Premadi et al. (1998):

we treat the density dumps of a given simulation as different sub-volumes of the same past light cone.

To break the artificial correlation that exists across simulated volumes at different redshifts, we perform

a rotation of each box plus a random shift of its origin.

The next step consists in calculating the photon geodesics through the large scale structures, and

to compute the cumulative deformation acquired along each trajectory. The most accurate method

would propagate the photons in a full three-dimensional volume, with the distortion computed along the

deflected trajectory. It was shown that this is an overkill, and that calculating the distortion only at the

mid-planes of the box provides weak lensing fields that differ by no more than 0.1 per cent with the full

three dimensional treatment (Vale & White, 2003). Moreover, it was shown at the same time that this

simplification has nearly indistinguishable effects on the two- and three-point functions. There is thus

no need to store the full dark matter density field, a significant advantage when working with hundreds

of high resolution N-body simulations. For the same reason, we did not adopted the three-dimensional

lensing calculation proposed by Kiessling et al. (2011a), because it requires storing the particle catalogs

of the full past light cones. Many authors have since opted for such ray tracing or line-of-sight integration

techniques (Jain et al., 2000; Forero-Romero et al., 2007; Hilbert et al., 2009).

The first step in this approach consists in collapsing the cosmological sub-volumes into their mid-

planes, and calculating the geodesics on specified angular locations, or pixels, on these thin lenses. In

the weak lensing regime, these trajectories are close to straight lines, such that Born’s approximation

is very accurate (Schneider et al., 1998; White & Vale, 2004). In this paper, we therefore opt for a

line-of-sight integration along the unperturbed photon paths. We nevertheless stored the full simulated

lenses, allowing future analysis to test how much ray-tracing or post-Born calculations (Schneider et al.,

1998; Krause & Hirata, 2010) affect the results.

N-body computations in this setting are fast, but not optimal: the interpolation becomes very strong

at low redshift, thus increasing the impact of the simulation softening length. More over, a large portion

of the simulated volume is left unused: the past light cone, shaped like a truncated pyramid, is extracted

from a cuboid. One way to improve on these two effects is to reduce the size of the simulation box for

low redshift dumps, as done in White & Hu (2000) for example, who used six different box sizes to reach

sources at z = 1. That involved running six times more independent simulations, a price that is not

always affordable. In the current work, it would have been too expensive to run that many, but two


distinct volume sizes offers a reasonable trade off.

Since there is inevitably some wasted cosmological volume, we could in principle re-shuffle the pro-

jections axis and the origin to create about ten time more LOS from the same simulations, following

Jain et al. (2000)8. However, this would inevitably produce a small amount of extra correlation between

different realizations, which would propagate and contaminate the covariance matrix with extra non-

Gaussian features. We opted out from this option, however we saved the full mid-planes, for usages like

this to be available for future analyses requiring even larger statistics.

As mentioned in the introduction, this work is meant to outperform the dynamical range of previous

weak lensing simulations: we need sub-arc minute precision, with a field of view of a few degrees per

side. We design our LOS such that each pixel has an opening angle of 0.21 arc minute on each side, with

npix = 10242 pixels in total, for a total opening angle of 3.58o per side. Moreover, we constructed the

survey geometry such that rays shot at z = 0 will reach the edge of the small simulation box at z = 1.

This uniquely specifies the box size of our low-redshift simulations: with our choice of cosmology, we

get L = 147.0 h−1Mpc per side. Rays then enter higher redshifts boxes, which have a larger size. That

way, no unperturbed photon paths actually escape the volume yet. Our second requirement is that the

surface of the past light cone reaches the edge of the larger box at z = 2. This yields a comoving side

of L = 231.1 h−1Mpc. Some of the outer rays eventually leave the simulated volume at redshifts larger

than 2.0, in which case we enforce the periodicity of the simulations box(see Fig. 6.1). This situation

applies only to the last four lenses, hence the total amount of repeated structures is very small. This

is even further suppressed by the lensing kernel, which favours redshifts closer to z = 1 − 1.5, and by

the fact such high redshifts have fewer galaxies to start with. We could have avoided this ‘leakage’ by

choosing a larger volume for the high redshift boxes, but the resolution would have been penalized in a

critical region.

6.4.2 N-Body simulations

The N-body simulations are produced by CUBEP3M, an improved version of PMFAST (Merz et al.,

2005) that is both MPI and OPENMP parallel, memory local and also allows for particle-particle (pp)

interaction at the sub-grid level. 10243 particles are placed on a 20483 grid, and have their initial

displacements and velocities calculated from the Zel’dovich approximation (Zel’Dovich, 1970; Shandarin

& Zeldovich, 1989). The transfer function that enters this calculation was obtained from CAMB (Seljak

8We have used some of this wasted volume on occasions to extract lenses at low redshifts for some simulations thatcould not run until the end (reaching hard walltime limit, computer cluster downtime, growth of ultra dense regionsthat unbalance the work load, etc.). In that process, we made sure that no volume was used by more than one LOS byconstraining the random shifts to point to virgin domains. There is therefore no extra correlation induced.


z = 3 ... z = 2 ... z = 1 ... z = 0

147 Mpc/h231.1 Mpc/h

Figure 6.1: Geometry of the lines-of-sight. The global simulated volume consists of two adjacent rectangular prisms,

collapsed as a series of thin lenses. As explained in the text, high redshift volumes are larger, but the number of simulated

grid cells and pixels are kept constant. The observer sits at z = 0; the junction between the small (lower-z) and large

(higher-z) simulation boxes occurs at z = 1; the past light cone escapes the simulated volume beyond z = 2, where we

exploit the periodicity of the boundary condition to populate the edges; we store lenses and haloes up to z = 3.

& Zaldarriaga, 1996). We used the WMAP5 cosmology (Komatsu et al., 2009) in our simulations and

theoretical predictions : ΩΛ = 0.721, Ωm = 0.279, Ωb = 0.046, ns = 0.96, σ8 = 0.817 and h = 0.701.

This cosmology and the simulation volumes discussed above completely specify the mass of the particles:

in the large (small) boxes, we have mp = 1.2759× 109 (3.2837× 108) M⊙. Also, the comoving sub-grid

softening lengths rsoft are of 112.8 and 71.8 h−1kpc respectively9. At the transition between the two

volumes, this change in mass affects the mass function of the halo catalogues, since the smallest collapsed

structures do not have the same physical mass. However, the weak lensing signals rely only on contrasts

in grid projections, which keep no trace of sub-grid level objects.

The initial redshifts are selected such as to optimize both the run time and the accuracy of the

N-body code. These are chosen to be zi = 40.0 and 200.0 for the large and small boxes respectively. The

reason for selecting different starting redshifts resides in the fact that the smaller volumes are probing

smaller scales, hence they need to start earlier, at a time where the Nyquist frequency of the grid is well

in the linear regime. Each simulation is then evolved with CUBEP3M on 8 nodes of the TCS (Tightly

Coupled System) supercomputer at the SciNet HPC Consortium (Loken et al., 2010) – to which system

we dedicate the name of the simulation suite. The lens redshifts, zl, are found by breaking the comoving

distance between z = 0.0 and z = 1.0 into cubes of L = 147.0 h−1Mpc per side (and that between z = 1.0

9rsoft is defined to be a tenth of a grid cell, and is enforced by a sharp cutoff in the force of gravity for particle pairsseparated by smaller distances.


Table 6.1: Redshifts of the lenses. The projections for zl > 1.0 are produced with L = 231.1 h−1Mpc simulations, while

those for lower zl are obtained from an independent set of simulations with L = 147.0 h−1Mpc.

3.004 2.691 2.411 2.159 1.933 1.728 1.542

1.371 1.215 1.071 0.961 0.881 0.804 0.730

0.659 0.591 0.526 0.463 0.402 0.344 0.287

0.232 0.178 0.126 0.075 0.025

and z = 3.0 into cubes of L = 231.1 h−1Mpc per side), and solving for the redshift at each mid-planes.

The resulting redshifts are presented in Table 6.1. When the simulations reach these redshifts, the dark

matter particles are placed on to a 20483 grid following the ‘cloud-in-cell’ interpolation scheme (Hockney

& Eastwood, 1981), and the grid is then collapsed into a slab along a randomly selected axis.

With this configuration, we solve [Eq. 6.9] numerically for each pixel. We convert the χ integral

into a discrete sum at the lens locations χ(zl). The infinitesimal element dχ becomes dL/ngrid, where

ngrid = 2048 and L = 147.0 or 231.1 h−1Mpc, depending on the redshift of the lens. Under the single

source plane approximation, we can thus write the convergence field as

κ(x) =3H2

oΩm

2c2

zs∑

zl

δ(x)(1 + zl)χ(zl)(1− χ(zl)/χ(zs))dχ (6.12)

where δ(x) is the two-dimensional density contrast on the mid-plane10

6.5 Testing the simulations

In this section, we quantify the resolution and accuracy of the N-body simulations. We measure the

power spectrum of the three-dimensional density fields – i.e. before collapsing onto mid-planes, and

before interpolating on to the pixel locations of the past light cone – and extract the angular power

spectrum of the lines-of-sight. In both cases, we compare our results to non-linear predictions and

identify the limitations of our calculations.

6.5.1 Dark matter density power spectrum

The power spectrum of the matter density P (k) is a fast and informative test of the quality of the sim-

ulations. It probes the growth of structures at all scales available within the volume, and in comparison

10To avoid edge effects when computing the shear fields, we perform the Fourier transforms on the full periodic slabs,i.e. before the interpolation on to the lenses. As explained at the very end of section 6.3, one would otherwise have to zeropad the convergence fields.


with a reliable theoretical model, informs us about both the accuracy and the resolution limits of the

simulations. For a given density contrast δ(x), the power spectrum can be calculated from its Fourier

transform δ(k) as:

〈δ(k)δ(k′)〉 = (2π)3δD(k− k′)P (k) (6.13)

where the angle brackets refer to a volume average, and the Dirac delta function selects identical Fourier

modes. We present in Fig. 6.2 the power spectrum for our 185 simulations at two redshifts, z = 0.961

and z = 0.025. When compared with the theoretical predictions from CAMB (Lewis et al., 2000), we

observe that the lower redshift power is a few per cent lower for 0.4 < k < 1.0 hMpc−1, and that

the low (high) redshift over estimate the power by about ten (twenty) per cent for k > 1.0 hMpc−1.

These discrepancies can be caused by a number of effects, from finite box size to residual uncertainty

in the numerical integration, and inevitably propagate in the calculations of past light cone. However,

deviations from CAMB are observed in most N-body codes – Giocoli et al. (2010) measured up to 50 per

cent deviation in the Millenium simulation –, and do not affect qualitatively the final results, as long as

internal consistency is preserved11.

In N-body codes, resolution limits are mainly determined by the softening length in the gravitational

force, and typically cause an abrupt drop in the observed power spectrum at small scales. This drop of

power can be modelled, following (Vale & White, 2003), by a Gaussian filtering of the form exp[−k2σ2g ]

in the power spectrum, where σg = 0.155L/Ngrid. In our simulations, we observe that the structures

seem to be well modelled down to k = 10.0hMpc−1, which corresponds to a comoving length of about

630 h−1 kpc. It is possible to obtain a rough estimate of the impact of this resolution limit on the weak

lensing angular power spectrum as follow: we know that the redshift that dominates the lensing kernel

is about z ∼ 1.0, hence that at this distance, the Gaussian filter 1σg subtends an angle of θsoft = 0.148

arcsec. Of course, the signal is sensitive to structure much closer, where the softening angle becomes

much higher. This technique also depends on the details of the N-body code, and a more accurate

estimate of the resolution limit is found from a comparison of the simulated angular power spectrum

with a reliable non-linear model (see section 6.5.2).

When it comes to measuring the uncertainty on the matter power spectrum, most data analyses

worked in the framework of linear theory of structure formation. Notably, this presumes that different

Fourier modes of the matter density grow independently, such that the error bars on the power spec-

trum are well described by Gaussian statistics. For non-linear processes, however, the phases of different

11The version of CAMB that was used in our calculations does not incorporate recent corrections that were made inDecember 2011


10−1

100

101

100

102

104

P(k

)[(M

pc/h)3

]

10−1

100

101

−1

−0.5

0

0.5

1

k[h/Mpc]

∆P

/P

N−bodyCAMBCAMB + filter

z=0.025z=0.961

Figure 6.2: (top:) Power spectrum of 185 N-body simulations, at redshifts of 0.961 (bottom curve) and 0.025 (top curve).

The solid and dashed lines are the non-linear predictions, with and without the Gaussian filter respectively. The error bars

shown here are the standard deviation over our sampling. We observe a slight over estimate of power in the simulations for

scales smaller than k = 3.0hMpc−1. This is caused by a known loss of accuracy in the predictions, as one progresses deep in

the non-linear regime. At the same time, resolution effects are affecting these scales, with a turn over at k ∼ 10.0hMpc−1.

This regime is thus not to be trusted in terms of accuracy. (bottom:) Fractional error between the simulations and the

non-linear predictions.


Fourier modes start to couple together (Meiksin & White, 1999; Coles & Chiang, 2000; Chiang et al.,

2002), therefore higher order statistics, i.e. bispectrum, trispectrum, are then needed in order to com-

pletely characterize the field. Theoretical calculations can describe these quantities with a reasonable

accuracy, at least in the trans-linear regime, but N-body simulations provide the most accurate estimates

at all scales resolved. These simulations are primarily used to extract the non-Gaussian covariance ma-

trices of various weak lensing estimators, hence it is essential to pin down the sources non-Gaussian

features, and to monitor how these propagate in our calculations. Our plan is to organize the final

lensing estimators in about ten angular bins, hence 185 simulations are enough to ensure convergence

on each element of the covariance matrices.

The power spectrum covariance matrix is defined as

C(k, k′) = 〈P (k)− P (k)〉〈P (k′)− P (k′)〉 (6.14)

where the over-bar refers to the best estimate of the mean. The amount of correlation between different

scales is better visualized with the cross-correlation coefficient matrix, which is obtained from C(k, k′)

via

ρ(k, k′) =C(k, k′)√

C(k, k)C(k′, k′)(6.15)

and is shown for z = 0.961 in Fig. 6.3. We see that it is almost diagonal for large scales (low k), while

measurements become correlated as we progress towards smaller scales (higher k). At k ∼ 0.5hMpc−1,

for instance, the Fourier modes are more that 40 per cent correlated. At this redshift, these correlated

scales correspond to angles smaller than θ ∼ 18.35 arcmin on the sky, or to ℓ > 1180. We stress that

this correlation is due to the sole effect of non-linear dynamics, and is thus intrinsic to the density

fields. Recent results have shown that neglecting either the non-Gaussian nature of the uncertainty can

significantly underestimate the error on the power spectrum, even at scales traditionally considered as

linear (Ngan et al., 2012; Harnois-Deraps & Pen, 2012).

6.5.2 Weak lensing power spectrum

In order to understand the resolution of our lensing maps, we measure the angular power spectrum of the

κ(θ) fields, and compare the results with the non-linear predictions. These are obtained from a simple

modification of the CAMB package (Seljak & Zaldarriaga, 1996), in which the redshift window function

has been adjusted to match that of the current survey geometry. It uses Limber’s approximation to

express the convergence power spectrum as an integral over P (k). The power spectrum of the convergence


10−1

100

101

10−1

100

101

k[h/Mpc]

k[h

/M

pc]

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 6.3: Cross-correlation coefficient matrix of the density power spectrum, measured from of 185 N-body simulations,

at redshift of 0.961. Modes at k ∼ 0.5hMpc−1, corresponding to θ ∼ 18.35 arcmin, are more than 40 per cent correlated.

field is defined as:

〈κ(ℓ)κ(ℓ′)〉 = (2π)2δD(ℓ+ ℓ′)Cℓ (6.16)

where ℓ is the Fourier component corresponding to the real space vector θ. and where, again, the angle

brackets refer to an angle average. The convergence power spectrum, estimated from our simulations, is

shown in Fig. 6.4 where the error bars are the 1σ standard deviation on the sampling. When compared

with the non-linear model, we find a good agreement in lower multipoles, while the theoretical predictions

slightly underestimate the power for ℓ > 1000, consistent with the observations of Hilbert et al. (2009).

The strong departure at ℓ ∼ 30000 is caused by limitations in the resolution, and corresponds to an

angle of about 0.7 arcmin.

As mentioned earlier, the smallest angles of weak lensing observations are probing the non-linear

regime of the underlying density field, and it is known that the statistics describing the uncertainty on

Cℓ are non-Gaussian. As a matter of fact, Dore et al. (2009) have demonstrated that the non-Gaussian

features in the weak lensing fields lead to a significant loss of constraining power on the dark energy

equation of state12 (see figure 6 in their paper). For instance, at ℓ = 1000, the figure-of-merit differs by

50 per cent when compared to Gaussian calculations, and the difference is even larger when including

higher multipoles. Although most of the departures from Gaussianity in the data are currently lost in the

12Following the jargon developed in the Dark Energy Task Force (Albrecht et al., 2006), a lower ‘figure-of-merit’ corre-sponds to a larger error about the equation of state.


102

103

104

10−7

10−6

10−5

10−4

10−3

ℓ(ℓ+

1)C

ℓ/(2π)

102

103

104

−1

−0.5

0

0.5

1

1.5

ℓ

∆C

ℓ/C

ℓ

z=0.526z=0.961z=1.542z=3.004

Figure 6.4: (top:) Convergence power spectrum, measured from 185 N-body simulations, where the source redshift

distribution is a Dirac delta function at z ∼ 3.0, 1.5, 1.0 and 0.5 (top to bottom symbols). The solid lines correspond to the

non-linear predictions, which are calculated with a modification of the CAMB package (Seljak & Zaldarriaga, 1996), where

the sources and lenses have been placed according to the survey depth. The linear predictions at zs = 3 are represented

by the dashed line, and the error bars are the 1σ standard deviation over our sampling. We observe a slight over-estimate

of power in the simulations for z = 3 and ℓ > 1000 compared to non-linear predictions (solid line), and a more important

bias for lower redshifts. This is caused by an underestimate of the power spectrum in the theoretical predictions, which is

also visible in the smallest scales of the three dimensional dark matter power spectrum (i.e. Fig. 6.2). Similar trends are

observed in the Coyote Universe and SUNGLASS simulation suites. The low-ℓ power seems also to be in excess in the

simulations, however predictions are still with the error bars. (bottom:) Fractional error between the simulations and the

non-linear predictions.


103

104

103

104

ℓ

ℓ

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 6.5: Cross-correlation coefficient matrix of the (dimension-full) convergence power spectrum, measured from 185

LOS. We observe a strong correlation for ℓ of a few thousand, consistent with the findings of Dore et al. (2009).

observation noise, future lensing surveys are expected to improve enough on statistics and systematics

such that non-Gaussian features will become significant. The non-linear dynamics effectively correlate

the error bars in small scales, as seen in Fig. 6.5. As expected, we observe that all the multipoles with

ℓ > 1000 are more than 40 per cent correlated.

6.5.3 Halo Catalogues

This section briefly describes how the halo catalogues are created, and presents a few of their statistical

properties. We recall that one of our main objective is to construct mock galaxy catalogues on which

many secondary effects will be quantified. As mentioned before, we do not attempt to populate the

haloes in this paper, since this is a challenge on its own, and we wish to factor out this problem for

now. In the future, though, one could follow the strategy of Heymans et al. (2006), and populate

the haloes with galaxies under the conditional luminosity function of Cooray & Milosavljevic (2005),

then assign ellipticity following the elliptical or spiral model (Heavens et al., 2000; Heymans et al., 2004).

Alternatively, one could use the GALICS (Hatton et al., 2003) and MOMAF (Blaizot et al., 2005) pipelines

to create mock galaxy catalogues directly from our halo catalogues, following the prescription described

in Forero-Romero et al. (2007). Our plan is to incorporate, for the first time and in a systematic way,

the prescriptions for intrinsic alignment, source clustering, ellipticity-shear correlation, etc. (De Lucia

& Blaizot, 2007; Schneider & Bridle, 2010) all at once. This is crucial in order to interpret correctly the


signal from the data, which contains all these contributions.

In this work, haloes are constructed from the matter density fields with a spherical overdensity search

algorithm (Cole & Lacey, 1996). The first step is to assign the dark matter particles on a 20483 grid

with the Nearest Grid Point scheme (Hockney & Eastwood, 1981), and identify local density maxima.

The halo finder then ranks these candidates in decreasing order of peak height, and for those which are

above an inspection threshold value, it grows a spherical volume centred on the peak, computing for

each shell the integrated overdensity until it drops under the predicted critical value. The haloes that

are analyzed first are then removed from the density field, in preparation for the inspection of lower mass

candidates. This prevents particles from contributing multiple times, but at the same time limits the

resolution on sub-structures of the largest haloes. This is a mild cost for the purpose of these catalogues,

which are populated with low multiplicity of galaxies, and thus depend rather weakly on the sub-halo

structures. Finally, for each halo, we measure the mass, the centre-of-mass (CM) and peak positions, the

CM velocity, the velocity dispersion, the angular momentum in the CM frame, and the inertia matrix

σij , which allows us to use population algorithms that outperform those that depend solely on the halo

mass. Although the inertia matrix is biased by the fact that we are only searching for spherical regions,

we still recover significant information about the shape and orientation.

In all the plots of this section, we present properties of the haloes that populate the full simulation

box, even though, in the final mock catalogues, we keep only those that sit inside the past light cone. We

apply the same coordinate rotation and random shifting of the origin that was performed on the lenses,

such that the halo catalogues and the lenses trace the same underlying density field. To quantify the

accuracy of the halo catalogues, we first extract the power spectrum of the distribution, and compare

the results with the measurements from dark matter particles by computing the halo bias, defined as

b(k) =√Phalo(k)/Pparticle(k). From Fig. 6.6, we observe that both the shape and redshift dependence

agree generally well with the results from Iliev et al. (2011). A direct comparison is complicated, however,

since the bias is both redshift and mass dependent. We observe in Fig. 6.7 that at z = 1.071, the halo

mass function is in good agreement with both Press & Schechter (1974) and Sheth & Tormen (2002)

in the range 1 × 1011 − 2 × 1014M⊙. Higher redshifts seem to better fit Press-Schechter in the same

dynamical range (bottom-most curve in the upper panel) while lower redshifts are better described by

the Sheth-Tormen model. These are on the high side, however, and reach a positive bias of about 50

per cent for M > 1014M⊙.


10−1

100

101

100

101

k[h/Mpc]

b(k

)

Figure 6.6: Halo bias, for z = 0.025 (bottom curve) and z = 0.961 (top curve). These results are in good agreement with

the results of Iliev et al. (2011), even though the direct comparison is complicated by the fact that the bias is dependent

on both the redshift and the mass bins.

1010

1011

1012

1013

1014

10−25

10−20

10−15

10−10

dln

(N)/

dln

(M)

1010

1011

1012

1013

1014

0

0.5

1

1.5

2

M [M⊙]

Sim

/ST

HalosPSST

0.0251.0713.004

Figure 6.7: (top:) Halo mass function, compared to predictions, for redshift 0.025, 1.071 and 3.004 (top to bottom

curves). (bottom:) Ratio of the mass function to the theoretical predictions of Sheth-Tormen.


6.6 Weak Lensing with 2-Point Correlation Functions

Detection of a robust weak lensing signal from the data is a challenging task in itself for many rea-

sons. The number density of galaxies detected, with their shape resolved, needs to be high, and many

secondary effects, mentioned in the Introduction of this paper, contaminate the signal and need to be

either filtered out or controlled. Generally, different statistical estimators and filtering techniques are

sensitive to different scales, systematics and secondary effects, and their measurements correlate in a

unique way. It was recently shown in Vafaei et al. (2010) that the optimal approach for measurements

involving the cosmic shear and convergence depends on the survey geometry and on the cosmological

parameters investigated. For instance, the shear 2-point correlation function minimizes the correlation

across different angles, while mass aperture window statistics are more sensitive to smaller scales, hence

are better suited for surveys of limited coverage (Schneider et al., 1998).

Understanding the non-Gaussian aspects of these estimators is the goal of the next three sections.

For each of them, we first give a short description, then present their signal and associated non-Gaussian

uncertainty. The current section covers estimates based on the 2-point functions, section 6.7 discusses

window-integrated estimators, and section 6.8 describes alternative statistics based exclusively on the

convergence maps. In all cases, the theoretical predictions follow the prescriptions of Van Waerbeke

et al. (2001), which are third-order calculations based on perturbation theory.

The 2- and 3-point functions of the observed lensing field are known to provide a wealth of information

about the underlying density field. In this paper, we constructed 185 independent shear and convergence

maps from our N-body simulations. As described in section 6.3, these maps are extracted from the

projected density of the grid, hence information about individual particles is lost. To mimic the actual

detection from a galaxy survey, we Poisson sample each of the maps with 100000 random points and

construct mock catalogues, from which we extract the 2-point correlation function measurements. The

positions are purely random within the 12.84 deg2 patches, and the values at each point are interpolated

from the simulated grids. This completely bypasses more realistic galaxy population algorithm, which

will be addressed in future work.

6.6.1 Shear

In current weak lensing analyses, one of the strongest signal comes from a measurement of the 2-point

correlation function in the shear of galaxy, which is defined as:

ξij(θ) ≡ 〈γi(θ′)γj(θ + θ′)〉 (6.17)


where i and j refer to a pair of galaxies separated by angle θ = |θ|. In the absence of gravitational

lenses, ξij(θ) averages out to zero, hence a positive signal indicates a detection of cosmic shear. The

intrinsic distortion produced by a single massive object is exclusively aligned in the tangential direction

around its centre of mass. It is therefore natural to consider a coordinate system that is local for each

galaxy pairs, in which the tangential and rotated axes (t, r) are defined as the direction perpendicular

and parallel to the line joining them, respectively. The new components of the complex shear are written

as γ = γt + iγr. In fact, many of the weak lensing estimators can be simplified when expressed as a

function of these.

The corresponding correlation functions ξtt and ξrr are defined as the weighted average of the tan-

gential and rotated shears for pairs of galaxies separated by an angle θ=|xi − xj |, namely:

ξtt(θ) =Σwiwjγt(xi)γt(xj)

Σwiwj

(6.18)

ξrr(θ) =Σwiwjγr(xi)γr(xj)

Σwiwj

(6.19)

where the weights wi quantify how well the shear is measured on the object i. A convenient linear

combination of the tangential and rotated shears :

ξ±(θ) = ξtt ± ξrr (6.20)

is particularly useful since it is directly related to the convergence power spectrum:

ξ+(θ) =

∫ ∞

0

dℓ

2πℓCℓJ0(ℓθ) (6.21)

ξ−(θ) =

∫ ∞

0

dℓ

2πℓCℓJ4(ℓθ) (6.22)

where Jn(x) is the nth order first kind Bessel function. Hence measurements of ξrr,tt give a direct handle

on cosmological parameters that depend on Cℓ.

We show in Fig. 6.8 and 6.9 the 2-point correlation functions ξtt and ξrr respectively, as a function

of the separation angle and at 4 different redshifts. The error bars are the 1σ standard deviation on

the sampling, as estimated from our 185 lines-of-sight. The agreements between the simulations and the

theoretical predictions are well within the error bars down to 0.6 arcmin, which allows us to conclude

that the signals are well resolved at least in that range.

We next show in Fig. 6.10 the cross-correlation coefficient matrix related to the tt measurement, for

source redshifts at 3.0 and 1.0. These show that the error bars across different angles are at least 50 per

cent correlated for the highest redshift, and up to 80 per cent for lower redshift sources. This correlation


0

1

2

3

4

5

6

7

8x 10

−4

Z = 3.004

ξ tt 10

010

2

−0.6−0.4−0.2

00.2

0

0.5

1

1.5x 10

−4

Z = 0.961

100

102

−0.6−0.4−0.2

00.2

100

101

102

0

1

2

3

4

5x 10

−5

Z = 0.526

ξ tt

θ(arcmin)

100

102

−0.4−0.2

00.20.4

100

101

102

0

1

2

3

4

5

6x 10

−7

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.8: Shear correlation function ξtt, computed from Poisson sampling the simulated shear maps in the local (tt, rr)

coordinates, as described in the text. The solid line shows non-linear theoretical predictions on the mean of the estimators.

Theoretical predictions for Figs. 6-17 were obtained with third order expansion in perturbation theory, as described by

(Van Waerbeke et al., 2001). In each of these figures, top left to down right panels represent z = 3.004, 0.961, 0.526 and

0.025 respectively, and the inset is the fractional error between the predictions and the simulations. In this figure, the

fractional error between the mean and the theory is around 10 per cent, although the agreement is fully consistent with

1σ error bars.


0

1

2

3

4

5

6

7

8x 10

−4

Z = 3.004

ξ rr 10

0

−1.5

−1

−0.5

0

0

5

10

15x 10

−5

Z = 0.961

100

102

−1.5

−1

−0.5

0

100

101

102

0

1

2

3

4

5x 10

−5

Z = 0.526

ξ rr

θ(arcmin)

100

102

−1.5

−1

−0.5

0

0.5

100

101

102

0

1

2

3

4

5

6

7

8x 10

−7

Z = 0.075

θ(arcmin)

100

102

−1

−0.5

0

0.5

Figure 6.9: ξrr component. The fractional errors, shown in the insets, are larger than for the tt component by about

a factor of two, but still consistent at 1σ. This is encouraging in the sense that most estimators are based on the latter

quantity.


100

101

100

101

θ(arcmin)

θ(arcm

in)

0.5

0.6

0.7

0.8

0.9

1

100

101

100

101

θ(arcmin)

θ(arcm

in)

0.5

0.6

0.7

0.8

0.9

1

Figure 6.10: Cross-correlation coefficient matrix of the tt two-point function, with the source plane at z ∼ 3.0 (left) and

z ∼ 1.0 (right). These exhibit the strong correlation that exists between different angular bins.

becomes even stronger as the two angles become similar in size. Any robust results based on the

uncertainty about ξtt must therefore incorporate these off-diagonal components. Typically, calculations

that combine these measurements in a noise-weighted scheme will need to invert the full covariance

matrix, and the non-Gaussian error bars thus obtained will generally be smaller than a naive Gaussian

treatment. The rr matrices are qualitatively similar, to Fig. 6.10, hence we do not show them here13.

6.6.2 Convergence

Convergence – or magnification – has been successfully detected in recent data analyses (Scranton et al.,

2005; Hildebrandt et al., 2009, 2011). Although more challenging to measure, it serves as an important

cross-check of the shear results, plus it is theoretically cleaner: no (non-local) Fourier transforms are

needed in the reconstruction of the underlying dark matter density field. Following the procedure

developed in the last section, we calculate the κ auto-correlation function from Poisson sampling the

simulated convergence maps. In Fig. 6.11, we present our results and find that the agreement with

the non-linear theoretical predictions extends deep under the arc minute at all redshifts. The cross-

correlation coefficient matrices corresponding to these measurements are very similar to the γtt matrices,

hence we do not present the matrices here.

13The covariance matrices of many weak lensing estimators presented in this paper are visually similar, hence we do notpresent them all. They are nevertheless made available upon request.


0

5

10

15x 10

−4

Z = 3.004

ξ κκ 10

010

2−1.5

−1

−0.5

0

0.5

0

0.5

1

1.5

2

2.5

3x 10

−4

Z = 0.961

100

102

−1

−0.5

0

0.5

100

101

102

0

2

4

6

8

10x 10

−5

Z = 0.526

ξ κκ

θ(arcmin)

100

102

−1

−0.5

0

0.5

100

101

102

0

5

10

15x 10

−7

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.11: Convergence auto-correlation function ξκκ, at four different redshifts, constructed from Poisson samplings

the simulated κ-maps. Even though the means differ by about 20 per cent, as shown in the insets, the agreement with the

predictions is well within 1σ.


6.7 Weak Lensing with Window-Integrated Correlation Func-

tions

As mentioned earlier, the 2-point correlation functions are not always the best way to measure the cosmic

shear or convergence. Window-integrated correlation functions, such as the mass aperture variance, give

a second handle on many cosmological parameters (Schneider et al., 1998), and are also used in galaxy-

galaxy lensing and cluster lensing. The method consists in measuring the mean, the variance or even

higher-order moments of a given lensing field, which was beforehand convolved with a filter of variable

smoothening angle θ, generally either a ‘top hat’ or a compensated filter. The integrated results are

then computed as a function of θ.

When extracting such estimators from the shear signal, the choice of filter matters. One of the

advantage of the top hat filter is that it probes scales as large as the field of view, whereas compensated

filters are limited by a damping tail in the window shape that somehow wastes the boundaries of the

patch. Another advantage of the top hat filter is that it yields a signal-to-noise ratio that is optimal

for skewness measurements (Vafaei et al., 2010). On the other hand, a compensated mass aperture

filter is more sensitive to small scales, hence it does a better jobs at recovering the signal from surveys

with limited sky coverage. In addition, it is measured directly from the tangential shear field (Kaiser

& Squires, 1993; Schneider, 1996), hence is not affected by the systematic and statistical uncertainties

involved in the reconstruction of the convergence field.

The top hat filter is a circular aperture of radius θ, outside of which the signal is cut to zero. The

filtering process then measures the total shear in a filtered region of a map, γ, for a given opening angle

θ. This convolution is performed with Fourier transforms, and each of the maps are zero-padded in

order to reduce the effect of boundaries. We repeat such measurement over all maps and compute the

variance 〈|γ|2〉TH , which is related to the convergence power spectrum Cℓ as:

〈|γ2(θ)|〉TH =1

2π

∫dℓℓCℓWTH(ℓθ) (6.23)

with WTH(ℓθ) =4J2

1(ℓθ)

(ℓθ)2 (Kaiser, 1992). We compare our measurements with the non-linear predictions

in Fig. 6.12, as a function of θ, and find a good agreement at all redshift. There is a small bias in the

mean, which is nevertheless consistent within 1σ. The cross-correlation matrices associated with two of

these measurements are presented in Fig. 6.13 and exhibit the strong correlation that exists between

most angular bins.

We next consider a compensated aperture filter, which is constructed from the local tangential shear

mock catalogues. In this process, one of the galaxy in the pair is replaced by the centre of the filter.


0

5

10

15

x 10−4

Z = 3.004

〈γ2(θ

)〉T

H

100

−1

−0.5

0

0

0.5

1

1.5

2

2.5

3

3.5x 10

−4

Z = 0.961

100

−0.8−0.6−0.4−0.2

00.2

100

101

0

2

4

6

8

x 10−5

Z = 0.526

〈γ2(θ

)〉T

H

θ(arcmin)

100

−0.8−0.6−0.4−0.2

00.2

100

101

0

2

4

6

8

10

12x 10

−7

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.12: Top hat variance, 〈|γ|2〉, measured from shear maps, at 4 different redshifts. There is a small bias in the

mean of our measurements, especially at low redshift, however the predictions are still well within the error bars.

10−2

10−1

10−2

10−1

θ(arcmin)

θ(arcm

in)

0.5

0.6

0.7

0.8

0.9

1

10−2

10−1

10−2

10−1

θ(arcmin)

θ(arcm

in)

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 6.13: Cross-correlation coefficient matrix of the top hat variance, with the source plane fixed at z ∼ 3.0 (left)

and z ∼ 1.0 (right).


The aperture mass Map is then given by (Schneider et al., 1998):

Map(θ) =

∫d2ϑQθ(ϑ)γt(ϑ) (6.24)

where Q is a weight function with support |ϑ| ∈ [0, θ] and which takes the shape:

Qθ(ϑ) =6

πθ2

(ϑ

θ2

)(1− ϑ2

θ2

)(6.25)

We then calculate the variance 〈M2ap〉 across the map, for all available angles, which is also related to

the convergence power spectrum:

〈M2ap(θ)〉 =

1

2π

∫ ∞

0

dℓℓCℓWap(ℓθ) (6.26)

withWap(ℓθ) =276J2

4(ℓθ)

(ℓθ)4 . We present in Fig. 6.14 our measurements of 〈M2ap〉 from the simulations, as a

function of smoothing scale θ. Over-plotted are the theoretical predictions obtained from [Eq. 6.26]. We

observe that for z > 1.0, the agreement extends down to the arc minute, whereas lower redshifts suffer

from a lack of variance at angles of a few arc minutes. This is caused by limitations in the resolution due

to strong zooming from the simulation grid on to the pixel map. We recall that with compensated filters,

an opening angle θ really probes scales at an angle ∼ θ/5, which approach the simulation resolution at

very low redshifts. This drop is also expected from the top hat variance, but appears at much smaller

smoothing angles. The cross-correlation coefficient matrices are presented in Fig. 6.15, and show that

most measurements are close to 60 per cent correlated. The smallest angles probe scales that approach

the pixel resolution, hence there is very little cross-correlation.

6.8 Weak Lensing with Windowed Statistics on Convergence

Maps

Window statistics performed directly on the convergence fields serve as an important test of the accuracy

and precision of the simulations, since the calculations here can be done directly on the grid, i.e. without

the Poisson sampling. We smooth the κ-maps with filters identical to those used in the last section and

calculate the top hat variance 〈κ2(θ)〉TH and mass aperture variance 〈M2ap(θ)〉 as a function of the filter

opening angle. In the latter case, the choice of compensated filter ( i.e. the equivalent of Qθ(ϑ) in [Eq.

6.25] ) is given by Uθ(ϑ) (Schneider et al., 1998), where:

Uθ(ϑ) =9

πθ2

(1− ϑ

θ2

)(1

3− ϑ2

θ2

)(6.27)

The Map estimator is now obtained from the convergence maps as:

Map(θ) =

∫d2ϑUθ(ϑ)κ(ϑ) (6.28)


2

4

6

8

10

12

x 10−5

Z = 3.004

〈γ2(θ

)〉A

pr

100

−0.8−0.6−0.4−0.2

00.2

0.5

1

1.5

2

2.5

x 10−5

Z = 0.961

100

−0.8−0.6−0.4−0.2

00.2

100

101

0

1

2

3

4

5

6

7

8

x 10−6

Z = 0.526

〈γ2(θ

)〉A

pr

θ(arcmin)

100

−1

−0.5

0

0.5

100

101

0

1

2

3

4

5

6

7

x 10−8

Z = 0.075

θ(arcmin)

100

−1.5

−1

−0.5

0

Figure 6.14: Aperture mass variance 〈M2ap〉 measured from tangential shear maps. The apparent discrepancy between

simulations and theoretical predictions at low redshift is caused by resolution limits, where the smallest angles actually

probe scales that are approaching the grid size. The intrinsic pixel size of 0.21 arcmin correspond to angles of about 1.0

arcmin with this compensated filter.

10−2

10−1

10−2

10−1

θ(arcmin)

θ(arcm

in)

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10−2

10−1

10−2

10−1

θ(arcmin)

θ(arcm

in)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 6.15: Cross-correlation coefficient matrix of the aperture mass variance, measured from the tangential shear,

with the source plane fixed at z ∼ 3.0 (left) and z ∼ 1.0 (right). The correlation seems to vanish in the first bin, which is

approaching the simulation resolution limit.


0

0.5

1

1.5

2x 10

−3

Z = 3.004

〈κ2(θ

)〉T

H10

0

−1

−0.5

0

0

1

2

3

4

5x 10

−4

Z = 0.961

100

−1

−0.5

0

100

101

0

2

4

6

8

10x 10

−5

Z = 0.526

〈κ2(θ

)〉T

H

θ(arcmin)

100

−1.5

−1

−0.5

0

0.5

100

101

0

5

10

15

20

x 10−7

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.16: 〈|γ|2〉TH measured directly from the convergence maps. We see that the agreement with the theoretical

predictions is excellent at all redshifts. In absence of large systematic uncertainties, as prevails in simulated environments,

this figure is similar to Fig. 6.12.

This measurement is complimentary to the shear approach (Eq. 6.24), and will yield identical results if

the systematics are well understood.

We present in Fig. 6.16 and 6.17 our measurements of the top hat and mass aperture variance

respectively. We observe that the signals are almost identical with the corresponding shear estimators

(Fig. 6.12 and 6.14). The agreement with the predictions is good at all redshifts for the top hat variance,

with only a slight bias at the lowest redshifts, whereas the mass aperture variance shows a lack of signal

at angles of a few arc minutes for low redshifts, consistent with the shear results.

We finally show the three-point function 〈M3ap(θ)〉 and 〈κ3(θ)〉TH in Fig. 6.18 and 6.19, both mea-

sured directly on the convergence maps. We recall that these measurements are essential to break the

degeneracy between σ8 and Ωm. We observe a good agreement between the predictions and the top hat

measurements, whereas the simulations tend to overestimate the mass aperture predictions by 1σ at low

angles. This comes again from the fact that the aperture filter is sensitive to about one fifth of the total

opening angle probed. Hence the discrepancy observed at θ ∼ 2 arcmin is mainly probing scales of 0.4

arcmin, which is of the order of the pixel size.


2

4

6

8

10

12

x 10−5

Z = 3.004

〈κ2(θ

)〉A

pr

100

−0.5

0

0.5

0.5

1

1.5

2

2.5

3x 10

−5

Z = 0.961

100

−0.6−0.4−0.2

00.20.4

100

101

1

2

3

4

5

6

7

x 10−6

Z = 0.526

〈κ2(θ

)〉A

pr

θ(arcmin)

100

−0.5

0

0.5

100

101

0

2

4

6

8

x 10−8

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.17: Aperture mass variance 〈M2ap〉, measured directly from the convergence maps. We recall that the effect of

finite pixel size is felt at larger angular scales – up to about one arc minute – with this estimator. This figure is equivalent

to Fig. 6.14.


0

5

10

15

20x 10

−7

Z = 3.004

〈κ3(θ

)〉A

pr

100

−1

0

1

0

5

10

15

20x 10

−8

Z = 0.961

100

−1

0

1

100

101

0

0.5

1

1.5

2

2.5

3

3.5

x 10−8

Z = 0.526

〈κ3(θ

)〉A

pr

θ(arcmin)

100

102

−2

−1

0

1

100

101

−4

−2

0

2

4

6

8

10

x 10−11

Z = 0.075

θ(arcmin)

100

102

−2

−1

0

1

Figure 6.18: 〈M3ap〉 measured directly from the convergence maps. We recall that the effect of finite pixel size is felt

to larger angular scales – up to about one arc minute in lower redshift lenses – with this estimator, which explains the

damped tail at small angles in the z = 0.075 plot.


0

2

4

6

8

10

x 10−5

Z = 3.004

〈κ3(θ

)〉T

H

100

−1

0

1

0

5

10

15

x 10−6

Z = 0.961

100

−1

0

1

100

101

0

0.5

1

1.5

2

2.5

3

3.5

x 10−6

Z = 0.526

〈κ3(θ

)〉T

H

θ(arcmin)

100

−1

0

1

100

101

−5

0

5

10

15

20

x 10−9

Z = 0.075

θ(arcmin)

100

−1

0

1

Figure 6.19: 〈|γ|3〉TH measured directly from the convergence maps.


6.9 Conclusion

This paper has two principal objectives: 1) measure the non-Gaussian covariance matrix on the principal

weak lensing estimators with sub-arc minute precision , and 2) set the stage for systematic studies of

secondary effects, and especially how their combination impacts the lensing signal. We have generated

a set of 185 high resolution N-body simulations, the TCS simulation suite, from which we constructed

past light cones with a ray tracing algorithm. The weak lensing signal is accurately resolved from a

few degrees down to a fraction of an arc minute. Thanks to the large statistics, we have measured

non-Gaussian error bars on a variety of weak lensing estimators, including 2-point correlation functions

on shear and convergence maps, and window-integrated estimators such as the mass aperture. In each

case, we compared our results with non-linear theoretical predictions at a few redshifts and obtained a

good agreement, which testifies the quality of the simulations.

In addition, we measured the covariance matrices for each of these estimators, and we show that the

error bars between most angular measurements are at least 50 per cent correlated, with regions up to 90

per cent correlated, especially when the two angles become closer. These non-Gaussian, correlated, error

bars are essential for a correct estimate of many derived quantities – including cosmological parameters

like σ8, Ωm or w, which so far relied either on Gaussian assumptions, or on numerical estimates that

were not resolving the complete dynamical range. With the next generation of lensing survey, however,

these non-Gaussian error bars, which intrinsically deviate significantly from Gaussian prescriptions, are

expected to be resolved, therefore techniques such as those presented here will be required for robust

estimates.

We also generated a series of halo mock catalogues that are coupled to the gravitational lenses for

future independent studies of secondary signals and alternate tests of weak lensing estimators. Within

the CFHTLenS collaboration, these catalogues will be part of the CLONE project. Our near term goal

is to include effects such as intrinsic alignment, source clustering, etc. in a single galaxy population

algorithm and quantify their combined contribution. In addition, we plan to quantify the impact of

post-Born calculations on the non-Gaussian uncertainty and on the contamination by secondary signals.

Understanding the impact of all these effects is essential, for such systematic bias are likely to contribute

significantly to future surveys such as KiDS and Euclid.

Aspects not included in our simulation settings are baryon feedback effects (Semboloni et al., 2011a)

and dependence of the covariance matrices on the cosmological parameters (Eifler et al., 2009). While

the latter can be simply addressed by running additional simulations, the former requires hydrodynam-

ical simulations that implement simultaneously matter clustering at large angular scales and a proper


modelling of feedback effects.

Acknowledgments

The authors would like to thank Ue-Li Pen, Dmitri Pogosyan, Catherine Heymans, Fergus Simpson,

Elisabetta Semboloni, Hendrick Hildebrandt, Martin Kilbinger and Christopher Bonnett for useful dis-

cussions and comments on the manuscripts. LVW acknowledges the financial support of NSERC and

CIfAR, and JHD is grateful for the FQRNT scholarship under which parts of this research was con-

ducted. N-body computations were performed on the TCS supercomputer at the SciNet HPC Con-

sortium. SciNet is funded by: the Canada Foundation for Innovation under the auspices of Compute

Canada; the Government of Ontario; Ontario Research Fund - Research Excellence; and the University

of Toronto.

Chapter 7

Conclusion and Future Work

This thesis presents improvements in the calculation of the uncertainty on cosmological parameters that

are extracted from the study of large scale structures. Current approaches treat the density fields as

Gaussian in nature, an assumption that is valid only for the largest scales observed. When extracting

the baryonic acoustic oscillation signal from the power spectrum of a density field, it has been shown

from N-body simulations that a Gaussian treatment underestimate by 15-20 per cent the error bar on

the stretching. This thesis presents a general strategy to remove this bias in a galaxy survey.

Performing a non-Gaussian power spectrum analysis is challenging for a number of reasons:

1. We need to incorporate the non-linear mode coupling effect, which correlates the error bars about

the power spectrum for scales that have undergone gravitational collapse. For this purpose, the

full power spectrum covariance matrix needs to be accurately measured, and convergence on each

matrix element must be assessed. Numerical simulations serve as a powerful probe here, since the

number of realizations can be made large enough.

2. In a survey, the observed covariance matrix needs to be convolved with the selection function.

Since neither the matrix nor the selection function are isotropic, the angular dependence must

be included. Mode coupling is expected to be stronger for pairs that are closer in magnitude or

angular separation, but theory has not been fully tested. We therefore need to measure the angular

dependence of the power spectrum covariance matrix, C(k, k′, θ) from N-body simulations.

3. The convolution itself is challenging as it involves an integral over six dimensions for each pair

(k, k′).

4. Since there are many ways for a field to be non-Gaussian, it is possible that the non-Gaussian

175

Chapter 7. Conclusion and Future Work 176

features observed in the simulations depart from those of Nature. To avoid such biases, non-

Gaussian features should be estimated from data itself. We therefore need to develop a procedure

to estimate the non-Gaussian error bars internally.

5. Data are limited in the sense that there is only one Universe to estimate the non-Gaussian error

bars, whereas we can generate hundreds or even thousands of N-body simulations. Consequently,

we need to develop tools to cope with the noise in this context.

After a detailed description of CUBEP3M – one of the most performant N-body code – in Chapter

2, we address in Chapter 3 the issue of bias in the presence of a survey selection function. From a large

ensemble of N-body simulations, we measure C(k, k′, θ) for the first time and describe a novel technique

that incorporates the extracted non-Gaussian features into a full analysis pipeline, for an arbitrary survey

geometry and selection function. We tested our method on the public 2dFGRS selection function and

show that the discrepancy with current error estimates reaches an order magnitude by k ∼ 0.2hMpc−1, a

scale that is traditionally thought to be insensitive to Gaussian departures, and two orders of magnitude

by k ∼ 1.0hMpc−1. This suggests that the error bars about the power spectrum in the 2dFGRS survey

must be interpreted with care, and that the error analyses of other surveys should be revisited as well.

Chapter 4 describes how an optimal measurement of the covariance matrix, combined with noise

filtering techniques, can characterize the non-Gaussian features of the error bars on the matter power

spectrum with only a handful of observation fields. We show that we can recover the Fisher information

content about the amplitude of the power spectrum – a metric widely used in the BAO community –

within 20 per cent, and we further provide a recipe to reproduce such a measurement in an actual data

analysis.

The question of mode coupling can be approached from a different angle: what if the non-linear effects

could be reduced by transforming the observed fields into something more Gaussian? The recent analyses

of the BOSS survey incorporate a density reconstruction algorithm that estimates the gravitational

potential from the observed matter field and reverses the gravitational infall following linear theory.

This effectively takes the field back in time, in a state where a smaller fraction was part of collapsed

objects. In Chapter 5, we show how this reconstruction method can be combined with a non-linear

Wiener filter constructed in wavelet space to minimize the power spectrum error bars. Taking again the

Fisher information as a metric, we show that the combination of both methods can recover up to an

order of magnitude more information compared to the original field. It is thus a promising avenue to

optimize the constraints on dark energy in current and next data analyses.

The impact non-linear dynamics on dark energy is also felt in the realm of weak gravitational lensing


analyses, which measure two- and three-point functions of shear or magnification maps to extract a

wealth of information about cosmological parameters. Although current surveys such as the CFHT

and SDSS are very competitive, the instrumental and statistical errors are still important enough that

non-Gaussian features contribute only marginally to the total uncertainty. With future surveys such

as Euclid and the KiDS however, a naive Gaussian treatment of the data will produce a systematic

bias that might dominate the error budget. It is therefore important to begin developing analyses that

extend beyond this approximation. In addition, fake weak lensing signals produced by source clustering,

intrinsic alignment, etc., are currently contaminating the analyses, and have only been quantified on a

one-by-one basis. In Chapter 6, we start to address both of these issues by measuring the non-Gaussian

uncertainty on the most common weak lensing estimators, and, in that process, construct accurate halo

catalogues from which secondary effects will be tested all at once. This latter part remains to be done,

and the goal is to incorporate all known effects into a dedicated galaxy catalogue in order to analyze

and understand how these combine together.

At the time I am writing this thesis, I have been granted the opportunity to pursue my research

as a CITA National Fellow, working under the supervision of Ludovic van Waerbeke. Producing mock

galaxy catalogues for the KiDS, as well as improving our understanding of the weak lensing secondary

signals will be at the centre of my task. Room will be left, however, to pursue my research on both the

BAO error analyses and on further developments of CUBEP3M. For instance, an important aspect of

non-Gaussian analyses is to reproduce the results of Chapter 3, this time exclusively based on simulated

halo distributions, and to see how this compares with particles. If one wishes to follow the lines of

Chapter 3 – i.e. estimating the error externally – it will also be important to understand how the non-

Gaussian parameters vary across redshift, cosmology, volume, and how they propagate deeper in the

non-linear regime (results to appear relatively soon). Perhaps one of the largest challenge that remains,

however, is to include redshift space distortions. For this to happen, numerical convergence will need to

be revisited since the number of elements to measure rapidly increases with the number of angular bins.

I am suspecting one will need noise filtering techniques such as those explained in Chapter 4 in order to

ensure convergence on the full error matrix.

It is quite interesting to note that the three Universities I will be attached to over the next two

years – McGill University, the University of Toronto and the University of British Columbia – are all

involved in the development of CHIME1, a new telescope that aims at measuring the BAO signal from

the 21 cm line of Hydrogen with intensity mapping. Although it is too early to say, we are expecting

1www.phas.ubc.ca/∼halpern/chime/


the construction to converge rather quickly, and I would not be surprised to find myself involved in the

numerical aspect of this experiment, i.e. in the creation of mock catalogues, error analyses, forecasts,

etc. To conclude, I wish to say that have been enjoying this five years journey, and I am looking forward

to contribute to the scientific community with the expertise I developed during my Ph.D.

Bibliography

Abbott, P. 2005, Mathematica J, 9, 689

Acquaviva, V., Hajian, A., Spergel, D. N., & Das, S. 2008, Phys. Rev. D, 78, 043514, 0803.2236

Albrecht, A. et al. 2006, ArXiv Astrophysics e-prints, arXiv:astro-ph/0609591

Anderson, L., et al. 2012, ArXiv e-prints, 1203.6594

Bartelmann, M., & Schneider, P. 2001, Phys. Rep., 340, 291, arXiv:astro-ph/9912508

Bartolo, N., Komatsu, E., Matarrese, S., & Riotto, A. 2004, Phys. Rep., 402, 103, arXiv:astro-

ph/0406398

Beaulieu, J. P., et al. 2010, in Astronomical Society of the Pacific Conference Series, Vol. 430, Pathways

Towards Habitable Planets, ed. V. Coude Du Foresto, D. M. Gelino, & I. Ribas, 266–+, 1001.3349

Benıtez, N., et al. 2009, ApJ, 691, 241, 0807.0535

Benjamin, J. et al. 2007, MNRAS, 381, 702, arXiv:astro-ph/0703570

Bernardeau, F. 1998, A&A, 338, 375, arXiv:astro-ph/9712115

Bernardeau, F., Colombi, S., Gaztanaga, E., & Scoccimarro, R. 2002, Phys. Rep., 367, 1, arXiv:astro-

ph/0112551

Bernardeau, F., van Waerbeke, L., & Mellier, Y. 1997, A&A, 322, 1

Bertschinger, E. 1985, ApJS, 58, 39

Blaizot, J., Wadadekar, Y., Guiderdoni, B., Colombi, S. T., Bertin, E., Bouchet, F. R., Devriendt,

J. E. G., & Hatton, S. 2005, MNRAS, 360, 159, arXiv:astro-ph/0309305

Blake, C., et al. 2010, MNRAS, 406, 803, 1003.5721

179

BIBLIOGRAPHY 180

——. 2011, ArXiv e-prints (astro-ph/1105.2862), 1105.2862

Blandford, R. D., Saust, A. B., Brainerd, T. G., & Villumsen, J. V. 1991, MNRAS, 251, 600

Chiang, L.-Y., Coles, P., & Naselsky, P. 2002, MNRAS, 337, 488, arXiv:astro-ph/0207584

Cohn, J. D. 2006, New Astronomy, 11, 226, arXiv:astro-ph/0503285

Cole, S., & Lacey, C. 1996, MNRAS, 281, 716, arXiv:astro-ph/9510147

Cole, S., et al. 2005, MNRAS, 362, 505, arXiv:astro-ph/0501174

Coles, P., & Chiang, L. 2000, Nature, 406, 376, arXiv:astro-ph/0006017

Coles, P., & Jones, B. 1991, MNRAS, 248, 1

Colless, M., et al. 2003, ArXiv e-prints (astro-ph/0306581), arXiv:astro-ph/0306581

Cooray, A., & Milosavljevic, M. 2005, ApJ, 627, L89, arXiv:astro-ph/0504580

Cooray, A., & Sheth, R. 2002, Phys. Rep., 372, 1, arXiv:astro-ph/0206508

Couchman, H. M. P. 1991, ApJ, 368, L23

Couchman, H. M. P., Thomas, P. A., & Pearce, F. R. 1995, ApJ, 452, 797, arXiv:astro-ph/9409058

Datta, K. K., Friedrich, M. M., Mellema, G., Iliev, I. T., & Shapiro, P. R. 2012a, MNRAS, 424, 762,

1203.0517

Datta, K. K., Mellema, G., Mao, Y., Iliev, I. T., Shapiro, P. R., & Ahn, K. 2012b, MNRAS, 424, 1877,

1109.1284

Daubechies, I. 1992, Ten Lectures on Wavelets (C B M S - N S F Regional Conference Series in Applied

Mathematics) (Soc for Industrial & Applied Math)

De Lucia, G., & Blaizot, J. 2007, MNRAS, 375, 2, arXiv:astro-ph/0606519

Desjacques, V., & Seljak, U. 2010, Phys. Rev. D, 81, 023006, 0907.2257

Desjacques, V., Seljak, U., & Iliev, I. T. 2009, MNRAS, 396, 85, 0811.2748

Dewdney, P. E., Hall, P. J., Schilizzi, R. T., & Lazio, T. J. L. W. 2009, IEEE Proceedings, 97, 1482

Dore, O., Lu, T., & Pen, U.-L. 2009, ArXiv e-prints, 0905.0501

Drinkwater, M. J., et al. 2010, MNRAS, 401, 1429, 0911.4246

BIBLIOGRAPHY 181

Dubinski, J., Kim, J., Park, C., & Humble, R. 2004, NewA, 9, 111, arXiv:astro-ph/0304467

Dyer, C. C., & Ip, P. S. S. 1993, ApJ, 409, 60

Eifler, T., Schneider, P., & Hartlap, J. 2009, A&A, 502, 721, 0810.4254

Eisenstein, D. J., Seo, H., Sirko, E., & Spergel, D. N. 2007, ApJ, 664, 675, arXiv:astro-ph/0604362

Eisenstein, D. J., Seo, H.-j., Sirko, E., & Spergel, D. 2007, Astrophys. J., 664, 675, astro-ph/0604362

Eisenstein, D. J., Seo, H.-j., & White, M. J. 2006, ArXiv e-prints, astro-ph/0604361

Eisenstein, D. J., et al. 2005, ApJ, 633, 560, astro-ph/0501171

Eleftheriou, M., Fitch, B. G., Rayshubskiy, A., Ward, T. J. C., & Germain, R. S. 2005, IBM J. Res.

Dev., 49, 457

Fang, L.-Z. e., & Thews, R. L. e. 1998, Wavelets in Physics (Singapur: World Scientific)

Feldman, H. A., Kaiser, N., & Peacock, J. A. 1994, ApJ, 426, 23, arXiv:astro-ph/9304022

Fernandez, E. R., Iliev, I. T., Komatsu, E., & Shapiro, P. R. 2012, ApJ, 750, 20, 1112.2064

Fisher, R. A. 1935, Journal of the Royal Statistical Society, 98, 39

Forero-Romero, J. E., Blaizot, J., Devriendt, J., van Waerbeke, L., & Guiderdoni, B. 2007, MNRAS,

379, 1507, arXiv:astro-ph/0611932

Friedrich, M. M., Mellema, G., Alvarez, M. A., Shapiro, P. R., & Iliev, I. T. 2011, MNRAS, 413, 1353,

1006.2016

Frigo, M., & Johnson, S. 2005, in Proceedings of the IEEE, Vol. 93, 216–231

Gehrels, N. 2010, ArXiv e-prints, 1008.4936

Giocoli, C., Bartelmann, M., Sheth, R. K., & Cacciato, M. 2010, MNRAS, 408, 300, 1003.4740

Goldberg, D. M., & Spergel, D. N. 2000, in Astronomical Society of the Pacific Conference Series, Vol.

201, Cosmic Flows Workshop, ed. S. Courteau & J. Willick, 282–+, arXiv:astro-ph/9909057

Hamilton, A. J. S., Rimes, C. D., & Scoccimarro, R. 2006, MNRAS, 371, 1188, arXiv:astro-ph/0511416

Harnois-Deraps, J., Pen, U., Iliev, I. T., Merz, H., Emberson, J. D., & Desjacques, V. 2012a, ArXiv

e-prints (astro-ph/1208.5098), 1208.5098

BIBLIOGRAPHY 182

Harnois-Deraps, J., & Pen, U.-L. 2012, MNRAS, 423, 2288, 1109.5746

Harnois-Deraps, J., Vafaei, S., & Van Waerbeke, L. 2012b, ArXiv e-prints, 1202.2332

Harnois-Deraps, J., Yu, H.-R., Zhang, T.-J., & Pen, U.-L. 2012c, ArXiv e-prints, 1205.4989

Hartlap, J., Simon, P., & Schneider, P. 2007, A&A, 464, 399, arXiv:astro-ph/0608064

Hatton, S., Devriendt, J. E. G., Ninin, S., Bouchet, F. R., Guiderdoni, B., & Vibert, D. 2003, MNRAS,

343, 75, arXiv:astro-ph/0309186

Heavens, A., & Peacock, J. 1988, MNRAS, 232, 339

Heavens, A., Refregier, A., & Heymans, C. 2000, MNRAS, 319, 649, arXiv:astro-ph/0005269

Heymans, C., Brown, M., Heavens, A., Meisenheimer, K., Taylor, A., & Wolf, C. 2004, MNRAS, 347,

895, arXiv:astro-ph/0310174

Heymans, C., & CFHTLenS Collaboration. 2012, in American Astronomical Society Meeting Abstracts,

Vol. 219, American Astronomical Society Meeting Abstracts, 130.01

Heymans, C., White, M., Heavens, A., Vale, C., & van Waerbeke, L. 2006, MNRAS, 371, 750,

arXiv:astro-ph/0604001

Hilbert, S., Hartlap, J., & Schneider, P. 2011, A&A, 536, A85, 1105.3980

Hilbert, S., Hartlap, J., White, S. D. M., & Schneider, P. 2009, A&A, 499, 31, 0809.5035

Hildebrandt, H. et al. 2011, ApJ, 733, L30, 1103.4407

Hildebrandt, H., van Waerbeke, L., & Erben, T. 2009, A&A, 507, 683, 0906.1580

Hirata, C. M., & Seljak, U. 2004, Phys. Rev. D, 70, 063526, arXiv:astro-ph/0406275

Hockney, R. W., & Eastwood, J. W. 1981, Computer Simulation Using Particles

——. 1988, Computer simulation using particles (Bristol: Hilger, 1988)

Hoekstra, H., & Jain, B. 2008, Annual Review of Nuclear and Particle Science, 58, 99, 0805.0139

Hu, W., & Okamoto, T. 2002, ApJ, 574, 566, arXiv:astro-ph/0111606

Huchra, J. P., Geller, M. J., de Lapparent, V., & Corwin, Jr., H. G. 1990, ApJS, 72, 433

Huterer, D. 2002, Phys. Rev. D, 65, 063001, arXiv:astro-ph/0106399

BIBLIOGRAPHY 183

Hutsi, G. 2006, A&A, 449, 891, arXiv:astro-ph/0512201

Ilie, C., Freese, K., Valluri, M., Iliev, I. T., & Shapiro, P. R. 2012, MNRAS, 422, 2164

Iliev, I. T., Ahn, K., Koda, J., Shapiro, P. R., & Pen, U.-L. 2010, proceedings paper for Moriond 2010

meeting (ArXiv:1005.2502), 1005.2502

Iliev, I. T., Mellema, G., Pen, U.-L., Merz, H., Shapiro, P. R., & Alvarez, M. A. 2006, MNRAS, 369,

1625, astro-ph/0512187

Iliev, I. T., Mellema, G., Shapiro, P. R., Pen, U.-L., Mao, Y., Koda, J., & Ahn, K. 2011, ArXiv e-prints,

1107.4772

——. 2012, MNRAS, 423, 2222, 1107.4772

Iliev, I. T., & Shapiro, P. R. 2001, MNRAS, 325, 468, arXiv:astro-ph/0101067

Iliev, I. T., Shapiro, P. R., Mellema, G., Merz, H., & Pen, U.-L. 2008, in refereed proceedings of

TeraGrid08, ArXiv e-prints (0806.2887), 0806.2887

Jain, B., Seljak, U., & White, S. 2000, ApJ, 530, 547

Jarosik, N., et al. 2011, ApJS, 192, 14, 1001.4744

Jing, Y. P. 2005, ApJ, 620, 559, arXiv:astro-ph/0409240

Joachimi, B., Taylor, A. N., & Kiessling, A. 2011, ArXiv e-prints, 1104.1399

Kaiser, N. 1992, ApJ, 388, 272

Kaiser, N., & Squires, G. 1993, ApJ, 404, 441

Kiessling, A., Heavens, A. F., Taylor, A. N., & Joachimi, B. 2011a, MNRAS, 414, 2235, 1011.1476

Kiessling, A., Taylor, A. N., & Heavens, A. F. 2011b, MNRAS, 416, 1045, 1103.3245

Komatsu, E. et al. 2009, ApJS, 180, 330, 0803.0547

Komatsu, E., et al. 2011, ApJS, 192, 18, 1001.4538

Krause, E., & Hirata, C. M. 2010, A&A, 523, A28, 0910.3786

Lacey, C., & Cole, S. 1994, MNRAS, 271, 676, arXiv:astro-ph/9402069

Laureijs, R. et al. 2011, ArXiv e-prints, 1110.3193

BIBLIOGRAPHY 184

Lawrence, E., Heitmann, K., White, M., Higdon, D., Wagner, C., Habib, S., & Williams, B. 2010, ApJ,

713, 1322, 0912.4490

Lazio, J. 2008, in American Institute of Physics Conference Series, Vol. 1035, The Evolution of Galaxies

Through the Neutral Hydrogen Window, ed. R. Minchin & E. Momjian, 303–309

Lee, J., & Pen, U.-L. 2008, ApJ, 686, L1, 0807.1538

Lewis, A., Challinor, A., & Lasenby, A. 2000, Astrophys. J., 538, 473, astro-ph/9911177

Limber, D. N. 1953, ApJ, 117, 134

Loken, C., et al. 2010, J. Phys.: Conf. Ser., 256, 1

LSST Science Collaborations, et al. 2009, ArXiv e-prints (astro-ph/0912.0201), 0912.0201

Lu, T., & Pen, U.-L. 2008, MNRAS, 388, 1819, 0710.1108

Lu, T., Pen, U.-L., & Dore, O. 2010, Phys. Rev. D, 81, 123015, 0905.0499

Mao, Y., Shapiro, P. R., Mellema, G., Iliev, I. T., Koda, J., & Ahn, K. 2012, MNRAS, 422, 926,

1104.2094

McDonald, P., & Eisenstein, D. J. 2007, Phys. Rev. D, 76, 063009, arXiv:astro-ph/0607122

McQuinn, M., & White, M. 2011, MNRAS, 415, 2257, 1102.1752

Meiksin, A., & White, M. 1999, MNRAS, 308, 1179, arXiv:astro-ph/9812129

Mellema, G., Iliev, I. T., Alvarez, M. A., & Shapiro, P. R. 2006, New Astronomy, 11, 374, arXiv:astro-

ph/0508416

Merz, H., Pen, U.-L., & Trac, H. 2005, New Astronomy, 10, 393, arXiv:astro-ph/0402443

Navarro, J. F., Frenk, C. S., & White, S. D. M. 1997, ApJ, 490, 493

Neyrinck, M. C., & Szapudi, I. 2007, MNRAS, 375, L51, arXiv:astro-ph/0610211

Neyrinck, M. C., Szapudi, I., & Rimes, C. D. 2006, MNRAS, 370, L66, arXiv:astro-ph/0604282

Neyrinck, M. C., Szapudi, I., & Rimes, C. D. 2006, MNRAS, 370, L66, astro-ph/0604282

Neyrinck, M. C., Szapudi, I., & Szalay, A. S. 2009, ApJ, 698, L90, 0903.4693

BIBLIOGRAPHY 185

Ngan, W., Harnois-Deraps, J., Pen, U.-L., McDonald, P., & MacDonald, I. 2012, MNRAS, 419, 2949,

1106.5548

Noh, Y., White, M., & Padmanabhan, N. 2009, Phys. Rev. D, 80, 123501, 0909.1802

Norberg, P., et al. 2002, MNRAS, 336, 907, arXiv:astro-ph/0111011

Padmanabhan, N., White, M., & Cohn, J. D. 2009, Phys. Rev. D, 79, 063523, 0812.2905

Padmanabhan, N., Xu, X., Eisenstein, D. J., Scalzo, R., Cuesta, A. J., Mehta, K. T., & Kazin, E. 2012,

ArXiv e-prints, 1202.0090

Pang, B., Pen, U.-l., & Perrone, M. 2010, ArXiv e-prints, 1004.1680

Pen, U.-L. 1999, Royal Society of London Philosophical Transactions Series A, 357, 2561, arXiv:astro-

ph/9904170

Pen, U.-L., Arras, P., & Wong, S. 2003, ApJS, 149, 447, arXiv:astro-ph/0305088

Percival, W. J., Cole, S., Eisenstein, D. J., Nichol, R. C., Peacock, J. A., Pope, A. C., & Szalay, A. S.

2007, MNRAS, 381, 1053, 0705.3323

Percival, W. J., et al. 2001, MNRAS, 327, 1297, arXiv:astro-ph/0105252

Peterson, J. B., Bandura, K., & Pen, U. L. 2006, ArXiv e-prints (astro-ph/0606104), arXiv:astro-

ph/0606104

Planck Collaboration et al. 2011, ArXiv e-prints, 1101.2022

Pope, A. C., & Szapudi, I. 2007, ArXiv e-prints, arXiv/0711.2509

Premadi, P., Martel, H., & Matzner, R. 1998, ApJ, 493, 10, arXiv:astro-ph/9708129

Press, W., Teukolshy, S., Vetterling, W., & Flannery, B. 1992, Numerical Recipes in FORTRAN. The

second edition (Cambridge University Press, 1992)

Press, W. H., & Schechter, P. 1974, ApJ, 187, 425

Reed, D. S., Bower, R., Frenk, C. S., Jenkins, A., & Theuns, T. 2007, MNRAS, 374, 2, arXiv:astro-

ph/0607150

Rimes, C. D., & Hamilton, A. J. S. 2005, MNRAS, 360, L82, arXiv:astro-ph/0502081

——. 2006, MNRAS, 371, 1205, arXiv:astro-ph/0511418

BIBLIOGRAPHY 186

Sato, M., Hamana, T., Takahashi, R., Takada, M., Yoshida, N., Matsubara, T., & Sugiyama, N. 2009,

ApJ, 701, 945, 0906.2237

Sato, M., Takada, M., Hamana, T., & Matsubara, T. 2011a, ApJ, 734, 76, 1009.2558

Sato, T., Hutsi, G., & Yamamoto, K. 2011b, Progress of Theoretical Physics, 125, 187, 1010.0289

Schilizzi, R. T. 2007, Highlights of Astronomy, 14, 539

Schlegel, D., White, M., & Eisenstein, D. 2009a, in Astronomy, Vol. 2010, astro2010: The Astronomy

and Astrophysics Decadal Survey, 314–+, 0902.4680

Schlegel, D., White, M., & Eisenstein, D. 2009b, ArXiv e-prints (astro-ph/0902.4680), 0902.4680

Schneider, M. D., & Bridle, S. 2010, MNRAS, 402, 2127, 0903.3870

Schneider, P. 1996, MNRAS, 283, 837, arXiv:astro-ph/9601039

Schneider, P., van Waerbeke, L., Jain, B., & Kruse, G. 1998, MNRAS, 296, 873, arXiv:astro-ph/9708143

Scoccimarro, R., Sefusatti, E., & Zaldarriaga, M. 2004, Phys. Rev. D, 69, 103513, arXiv:astro-ph/0312286

Scoccimarro, R., & Sheth, R. K. 2002, MNRAS, 329, 629, arXiv:astro-ph/0106120

Scranton, R. et al. 2005, ApJ, 633, 589, arXiv:astro-ph/0504510

Sefusatti, E., Crocce, M., & Desjacques, V. 2010, MNRAS, 406, 1014, 1003.0007

——. 2011, ArXiv e-prints, 1111.6966

Seljak, U., & Zaldarriaga, M. 1996, ApJ, 469, 437, arXiv:astro-ph/9603033

Semboloni, E., Heymans, C., van Waerbeke, L., & Schneider, P. 2008, MNRAS, 388, 991, 0802.3978

Semboloni, E., Hoekstra, H., Schaye, J., van Daalen, M. P., & McCarthy, I. G. 2011a, MNRAS, 417,

2020, 1105.1075

Semboloni, E. et al. 2006, A&A, 452, 51, arXiv:astro-ph/0511090

Semboloni, E., Schrabback, T., van Waerbeke, L., Vafaei, S., Hartlap, J., & Hilbert, S. 2011b, MNRAS,

410, 143, 1005.4941

Seo, H.-J., & Eisenstein, D. J. 2003, Astrophys. J., 598, 720, astro-ph/0307460

——. 2005, Astrophys. J., 633, 575, astro-ph/0507338

BIBLIOGRAPHY 187

——. 2007, ArXiv e-prints, astro-ph/0701079

Seo, H.-J., Sato, M., Dodelson, S., Jain, B., & Takada, M. 2011, ApJ, 729, L11+, 1008.0349

Shandarin, S. F., & Zeldovich, Y. B. 1989, Reviews of Modern Physics, 61, 185

Shapiro, P. R., Iliev, I. T., & Raga, A. C. 1999, MNRAS, 307, 203

Sheldon, E. S. et al. 2009, ApJ, 703, 2232, 0709.1162

Sheth, R. K., & Tormen, G. 2002, MNRAS, 329, 61, arXiv:astro-ph/0105113

Smith, R. E. et al. 2003, MNRAS, 341, 1311, arXiv:astro-ph/0207664

Springel, V. 2005, MNRAS, 364, 1105, arXiv:astro-ph/0505010

——. 2009, PiTP Summer School 2009, http://www.sns.ias.edu/pitp/2009files/Lecture Notes/

Springel lecture2%5B1%5D.pdf

Springel, V., Yoshida, N., & White, S. D. M. 2001, New Astronomy, 6, 79, arXiv:astro-ph/0003162

Takada, M., & Jain, B. 2009, MNRAS, 395, 2065, 0810.4170

Takahashi, R., et al. 2009, ApJ, 700, 479, 0902.0371

——. 2011, ApJ, 726, 7, 0912.1381

Taruya, A., Koyama, K., & Matsubara, T. 2008, Phys. Rev. D, 78, 123534, 0808.4085

Tegmark, M. 1997, Phys. Rev. Lett., 79, 3806, astro-ph/9706198

Tegmark, M., Taylor, A. N., & Heavens, A. F. 1997, ApJ, 480, 22, arXiv:astro-ph/9603021

Tegmark, M., et al. 2006, Phys. Rev. D, 74, 123507, arXiv:astro-ph/0608632

Tinker, J., Kravtsov, A. V., Klypin, A., Abazajian, K., Warren, M., Yepes, G., Gottlober, S., & Holz,

D. E. 2008, ApJ, 688, 709, 0803.2706

Trac, H., & Cen, R. 2007, ApJ, 671, 1, arXiv:astro-ph/0612406

Trac, H., & Pen, U. 2003, in Bulletin of the American Astronomical Society, Vol. 35, American Astro-

nomical Society Meeting Abstracts, 1365

Trac, H., & Pen, U.-L. 2004, NewA, 9, 443, arXiv:astro-ph/0309599

BIBLIOGRAPHY 188

Vafaei, S., Lu, T., van Waerbeke, L., Semboloni, E., Heymans, C., & Pen, U.-L. 2010, Astroparticle

Physics, 32, 340, 0905.3726

Vale, C., & White, M. 2003, ApJ, 592, 699, arXiv:astro-ph/0303555

Van Waerbeke, L., Hamana, T., Scoccimarro, R., Colombi, S., & Bernardeau, F. 2001, MNRAS, 322,

918, arXiv:astro-ph/0009426

Vogeley, M. S., & Szalay, A. S. 1996, ApJ, 465, 34, arXiv:astro-ph/9601185

Weinberg, D. H. 1992, MNRAS, 254, 315

White, M., & Hu, W. 2000, ApJ, 537, 1, arXiv:astro-ph/9909165

White, M., & Vale, C. 2004, Astroparticle Physics, 22, 19, arXiv:astro-ph/0312133

Xu, G. 1995, ApJS, 98, 355, arXiv:astro-ph/9409021

York, D. G., et al. 2000, AJ, 120, 1579, arXiv:astro-ph/0006396

Yu, H.-R., Harnois-Deraps, J., Zhang, T.-J., & Pen, U.-L. 2012, MNRAS, 421, 832, 1012.0444

Yu, Y., Zhang, P., Lin, W., Cui, W., & Fry, J. N. 2011, Phys. Rev. D, 84, 023523, 1103.2858

Zackrisson, E. et al. 2010, MNRAS, 407, L74, 1006.0481

Zahn, O., Lidz, A., McQuinn, M., Dutta, S., Hernquist, L., Zaldarriaga, M., & Furlanetto, S. R. 2007,

ApJ, 654, 12, astro-ph/0604177

Zahn, O., & Zaldarriaga, M. 2006, ApJ, 653, 922, arXiv:astro-ph/0511547

Zel’Dovich, Y. B. 1970, A&A, 5, 84

Zhang, T., Pen, U., Zhang, P., & Dubinski, J. 2003, ApJ, 598, 818

Zhang, T.-J., Yu, H.-R., Harnois-Deraps, J., MacDonald, I., & Pen, U.-L. 2011, ApJ, 728, 35, 1008.3506

Date post:	16-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

tspace.library.utoronto.ca...Dedication A Katam et Zia, qui font toute la diﬀ´erence, a...

Documents