Numerical Methods for Computational Science and Engineering

NumCSE, AT15, Prof. Ralf Hiptmair cSAM, ETH Zurich, 2015

ETH Lecture 401-0663-00L Numerical Methods for CSE

Numerical Methods forComputational Science and Engineering

Prof. R. Hiptmair, SAM, ETH Zurich

(with contributions from Prof. P. Arbenz and Dr. V. Gradinaru)

Autumn Term 2016

(C) Seminar fr Angewandte Mathematik, ETH Zrich

URL: https://people.math.ethz.ch/~grsam/HS16/NumCSE/NumCSE16.pdf

Always under construction!

SVN revision 90954

The online version will always be work in progress and subject

to change.

(Nevertheless, structure and main contents can be expected to

be stable)

Do not print!

Main source of information: Lecture homepage

Important links:

Lecture Git repository: https://gitlab.math.ethz.ch/NumCSE/NumCSE.git(Clone this repository to get access to most of the C++ codes in the lecture document and

homework problems. Git guide)

Lecture recording: http://www.video.ethz.ch/lectures/d-math/2016/autumn/401-0663-00L.html Tablet notes: http://www.sam.math.ethz.ch/~grsam/HS16/NumCSE/NCSE16_Notes/ Homework problems: https://people.math.ethz.ch/~grsam/HS16/NumCSE/NCSEProblems.pdf

, 1
https://people.math.ethz.ch/~grsam/HS16/NumCSE/NumCSE16.pdfhttps://people.math.ethz.ch/~grsam/HS16/NumCSE/https://git-scm.com/https://gitlab.math.ethz.ch/NumCSE/NumCSE.githttps://gitlab.math.ethz.ch/tille/gitlab-introduction/blob/master/git/README.mdhttp://www.video.ethz.ch/lectures/d-math/2016/autumn/401-0663-00L.htmlhttp://www.sam.math.ethz.ch/\char 126\relax grsam/HS16/NumCSE/NCSE16_Notes/https://people.math.ethz.ch/~grsam/HS16/NumCSE/NCSEProblems.pdf

Contents

0 Introduction 8

0.0.1 Focus of this course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

0.0.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

0.0.3 To avoid misunderstandings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110.0.4 Reporting errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

0.0.5 Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

0.1 Specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

0.1.1 Assistants and exercise classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

0.1.2 Study center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

0.1.3 Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

0.1.4 Information on examinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

0.2 Programming in C++11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

0.2.1 Function Arguments and Overloading . . . . . . . . . . . . . . . . . . . . . . . . . 20

0.2.2 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

0.2.3 Function Objects and Lambda Functions . . . . . . . . . . . . . . . . . . . . . . . 23

0.2.4 Multiple Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

0.2.5 A Vector Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

0.3 Creating Plots with MATHGL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

0.3.1 MATHGL Documentation (by J. Gacon) . . . . . . . . . . . . . . . . . . . . . . . . 38

0.3.2 MATHGL Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

0.3.3 Corresponding Plotting functions of MATLAB and MATHGL . . . . . . . . . . . . . . 39

0.3.4 The Figure Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

0.3.4.1 Introductory example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

0.3.4.2 Figure Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

1 Computing with Matrices and Vectors 54

1.1 Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

1.1.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

1.1.2 Classes of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

1.2 Software and Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

1.2.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

1.2.2 PYTHON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

1.2.3 EIGEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

1.2.4 (Dense) Matrix storage formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

1.3 Basic linear algebra operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

1.3.1 Elementary matrix-vector calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

1.3.2 BLAS Basic Linear Algebra Subprograms . . . . . . . . . . . . . . . . . . . . . 84

1.4 Computational effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

1.4.1 (Asymptotic) complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

1.4.2 Cost of basic operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

1.4.3 Reducing complexity in numerical linear algebra: Some tricks . . . . . . . . . . . . 97

2


1.5 Machine Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

1.5.1 Experiment: Loss of orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

1.5.2 Machine Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

1.5.3 Roundoff errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

1.5.4 Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

1.5.5 Numerical stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

2 Direct Methods for Linear Systems of Equations 135

2.1 Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

2.2 Theory: Linear systems of equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

2.2.1 Existence and uniqueness of solutions . . . . . . . . . . . . . . . . . . . . . . . . 138

2.2.2 Sensitivity of linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

2.3 Gaussian Ellimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

2.3.1 Basic algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

2.3.2 LU-Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

2.3.3 Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

2.4 Stability of Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

2.5 Survey: Elimination solvers for linear systems of equations . . . . . . . . . . . . . . . . . 174

2.6 Exploiting Structure when Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . 178

2.7 Sparse Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

2.7.1 Sparse matrix storage formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

2.7.2 Sparse matrices in MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

2.7.3 Sparse matrices in EIGEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

2.7.4 Direct Solution of Sparse Linear Systems of Equations . . . . . . . . . . . . . . . . 201

2.7.5 LU-factorization of sparse matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 205

2.7.6 Banded matrices [?, Sect. 3.7] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

2.8 Stable Gaussian elimination without pivoting . . . . . . . . . . . . . . . . . . . . . . . . . 219

3 DIrect Methods for Linear Least Squares Problems 226

3.0.1 Overdetermined Linear Systems of Equations: Examples . . . . . . . . . . . . . . 227

3.1 Least Squares Solution Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

3.1.1 Least Squares Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

3.1.2 Normal Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

3.1.3 Moore-Penrose Pseudoinverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

3.1.4 Sensitivity of Least Squares Problem . . . . . . . . . . . . . . . . . . . . . . . . . 238

3.2 Normal Equation Methods [?, Sect. 4.2], [?, Ch. 11] . . . . . . . . . . . . . . . . . . . . . 240

3.3 Orthogonal Transformation Methods [?, Sect. 4.4.2] . . . . . . . . . . . . . . . . . . . . . 243

3.3.1 Transformation Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

3.3.2 Orthogonal/Unitary Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

3.3.3 QR-Decomposition [?, Sect. 13], [?, Sect. 7.3] . . . . . . . . . . . . . . . . . . . . 245

3.3.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

3.3.3.2 Computation of QR-Decomposition . . . . . . . . . . . . . . . . . . . . . 248

3.3.3.3 QR-Decomposition: Stability . . . . . . . . . . . . . . . . . . . . . . . . . 254

3.3.3.4 QR-Decomposition in EIGEN . . . . . . . . . . . . . . . . . . . . . . . . . 256

3.3.4 QR-Based Solver for Linear Least Squares Problems . . . . . . . . . . . . . . . . 258

3.3.5 Modification Techniques for QR-Decomposition . . . . . . . . . . . . . . . . . . . . 263

3.3.5.1 Rank-1 Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

3.3.5.2 Adding a Column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

3.3.5.3 Adding a Row . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

3.4 Singular Value Decomposition (SVD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

3.4.1 SVD: Definition and Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

3.4.2 SVD in EIGEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

CONTENTS, CONTENTS 3


3.4.3 Generalized Solutions of LSE by SVD . . . . . . . . . . . . . . . . . . . . . . . . . 274

3.4.4 SVD-Based Optimization and Approximation . . . . . . . . . . . . . . . . . . . . . 276

3.4.4.1 Norm-Constrained Extrema of Quadratic Forms . . . . . . . . . . . . . . 277

3.4.4.2 Best Low-Rank Approximation . . . . . . . . . . . . . . . . . . . . . . . . 279

3.4.4.3 Principal Component Data Analysis (PCA) . . . . . . . . . . . . . . . . . 283

3.5 Total Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

3.6 Constrained Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

3.6.1 Solution via Lagrangian Multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . 301

3.6.2 Solution via SVD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

4 Filtering Algorithms 306

4.1 Discrete convolutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

4.2 Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

4.2.1 Discrete Convolution via DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

4.2.2 Frequency filtering via DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

4.2.3 Real DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

4.2.4 Two-dimensional DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

4.2.5 Semi-discrete Fourier Transform [?, Sect. 10.11] . . . . . . . . . . . . . . . . . . . 339

4.3 Fast Fourier Transform (FFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

4.4 Trigonometric transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

4.4.1 Sine transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

4.4.2 Cosine transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

4.5 Toeplitz Matrix Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

4.5.1 Toeplitz Matrix Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

4.5.2 The Levinson Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

5 Data Interpolation and Data Fitting in 1D 372

5.1 Abstract interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

5.2 Global Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

5.2.1 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

5.2.2 Polynomial Interpolation: Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

5.2.3 Polynomial Interpolation: Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 385

5.2.3.1 Multiple evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

5.2.3.2 Single evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

5.2.3.3 Extrapolation to zero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

5.2.3.4 Newton basis and divided differences . . . . . . . . . . . . . . . . . . . . 396

5.2.4 Polynomial Interpolation: Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . 401

5.3 Shape preserving interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

5.3.1 Shape properties of functions and data . . . . . . . . . . . . . . . . . . . . . . . . 405

5.3.2 Piecewise linear interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

5.4 Cubic Hermite Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

5.4.1 Definition and algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

5.4.2 Local monotonicity preserving Hermite interpolation . . . . . . . . . . . . . . . . . 414

5.5 Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417

5.5.1 Cubic spline interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

5.5.2 Structural properties of cubic spline interpolants . . . . . . . . . . . . . . . . . . . 422

5.5.3 Shape Preserving Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 425

5.6 Trigonometric Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432

5.6.1 Trigonometric Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

5.6.2 Reduction to Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 434

5.6.3 Equidistant Trigonometric Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 436

5.7 Least Squares Data Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440



6 Approximation of Functions in 1D 448

6.1 Approximation by Global Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

6.1.1 Polynomial approximation: Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

6.1.2 Error estimates for polynomial interpolation . . . . . . . . . . . . . . . . . . . . . . 457

6.1.3 Chebychev Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

6.1.3.1 Motivation and definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

6.1.3.2 Chebychev interpolation error estimates . . . . . . . . . . . . . . . . . . 472

6.1.3.3 Chebychev interpolation: computational aspects . . . . . . . . . . . . . . 477

6.2 Mean Square Best Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

6.2.1 Abstract theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

6.2.1.1 Mean square norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

6.2.1.2 Normal equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483

6.2.1.3 Orthonormal bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

6.2.2 Polynomial mean square best approximation . . . . . . . . . . . . . . . . . . . . . 486

6.3 Uniform Best Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493

6.4 Approximation by Trigonometric Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 497

6.4.1 Approximation by Trigonometric Interpolation . . . . . . . . . . . . . . . . . . . . . 497

6.4.2 Trigonometric interpolation error estimates . . . . . . . . . . . . . . . . . . . . . . 498

6.4.3 Trigonometric Interpolation of Analytic Periodic Functions . . . . . . . . . . . . . . 503

6.5 Approximation by piecewise polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 506

6.5.1 Piecewise polynomial Lagrange interpolation . . . . . . . . . . . . . . . . . . . . . 507

6.5.2 Cubic Hermite interpolation: error estimates . . . . . . . . . . . . . . . . . . . . . 511

6.5.3 Cubic spline interpolation: error estimates [?, Ch. 47] . . . . . . . . . . . . . . . . 515

7 Numerical Quadrature 520

7.1 Quadrature Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522

7.2 Polynomial Quadrature Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525

7.3 Gauss Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528

7.4 Composite Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542

7.5 Adaptive Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550

8 Iterative Methods for Non-Linear Systems of Equations 558

8.1 Iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560

8.1.1 Speed of convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562

8.1.2 Termination criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567

8.2 Fixed Point Iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570

8.2.1 Consistent fixed point iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571

8.2.2 Convergence of fixed point iterations . . . . . . . . . . . . . . . . . . . . . . . . . 572

8.3 Finding Zeros of Scalar Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578

8.3.1 Bisection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579

8.3.2 Model function methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580

8.3.2.1 Newton method in scalar case . . . . . . . . . . . . . . . . . . . . . . . 581

8.3.2.2 Special one-point methods . . . . . . . . . . . . . . . . . . . . . . . . . . 583

8.3.2.3 Multi-point methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587

8.3.3 Asymptotic efficiency of iterative methods for zero finding . . . . . . . . . . . . . . 592

8.4 Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594

8.4.1 The Newton iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594

8.4.2 Convergence of Newtons method . . . . . . . . . . . . . . . . . . . . . . . . . . . 605

8.4.3 Termination of Newton iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607

8.4.4 Damped Newton method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609

8.4.5 Quasi-Newton Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613

8.5 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618



8.5.1 Minima and minimizers: Some theory . . . . . . . . . . . . . . . . . . . . . . . . . 618

8.5.2 Newtons method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618

8.5.3 Descent methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618

8.5.4 Quasi-Newton methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618

8.6 Non-linear Least Squares [?, Ch. 6] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618

8.6.1 (Damped) Newton method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620

8.6.2 Gauss-Newton method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621

8.6.3 Trust region method (Levenberg-Marquardt method) . . . . . . . . . . . . . . . . . 624

9 Eigenvalues 628

9.1 Theory of eigenvalue problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632

9.2 Direct Eigensolvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634

9.3 Power Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638

9.3.1 Direct power method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638

9.3.2 Inverse Iteration [?, Sect. 7.6], [?, Sect. 5.3.2] . . . . . . . . . . . . . . . . . . . . 648

9.3.3 Preconditioned inverse iteration (PINVIT) . . . . . . . . . . . . . . . . . . . . . . . 660

9.3.4 Subspace iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663

9.3.4.1 Orthogonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668

9.3.4.2 Ritz projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673

9.4 Krylov Subspace Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678

10 Krylov Methods for Linear Systems of Equations 692

10.1 Descent Methods [?, Sect. 4.3.3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

10.1.1 Quadratic minimization context . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

10.1.2 Abstract steepest descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694

10.1.3 Gradient method for s.p.d. linear system of equations . . . . . . . . . . . . . . . . 695

10.1.4 Convergence of the gradient method . . . . . . . . . . . . . . . . . . . . . . . . . 697

10.2 Conjugate gradient method (CG) [?, Ch. 9], [?, Sect. 13.4], [?, Sect. 4.3.4] . . . . . . . . . 700

10.2.1 Krylov spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701

10.2.2 Implementation of CG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 702

10.2.3 Convergence of CG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705

10.3 Preconditioning [?, Sect. 13.5], [?, Ch. 10], [?, Sect. 4.3.5] . . . . . . . . . . . . . . . . . . 711

10.4 Survey of Krylov Subspace Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717

10.4.1 Minimal residual methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717

10.4.2 Iterations with short recursions [?, Sect. 4.5] . . . . . . . . . . . . . . . . . . . . . 718

11 Numerical Integration Single Step Methods 722

11.1 Initial value problems (IVP) for ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722

11.1.1 Modeling with ordinary differential equations: Examples . . . . . . . . . . . . . . . 723

11.1.2 Theory of initial value problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 727

11.1.3 Evolution operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 731

11.2 Introduction: Polygonal Approximation Methods . . . . . . . . . . . . . . . . . . . . . . . 733

11.2.1 Explicit Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734

11.2.2 Implicit Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736

11.2.3 Implicit midpoint method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737

11.3 General single step methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

11.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

11.3.2 Convergence of single step methods . . . . . . . . . . . . . . . . . . . . . . . . . 741

11.4 Explicit Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747

11.5 Adaptive Stepsize Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755

12 Single Step Methods for Stiff Initial Value Problems 770



12.1 Model problem analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771

12.2 Stiff Initial Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784

12.3 Implicit Runge-Kutta Single Step Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 789

12.3.1 The implicit Euler method for stiff IVPs . . . . . . . . . . . . . . . . . . . . . . . . 790

12.3.2 Collocation single step methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791

12.3.3 General implicit RK-SSMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794

12.3.4 Model problem analysis for implicit RK-SSMs . . . . . . . . . . . . . . . . . . . . . 796

12.4 Semi-implicit Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 802

12.5 Splitting methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805

13 Structure Preserving Integration [?] 811

Index 811

Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 823

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824


Chapter 0

Introduction

0.0.1 Focus of this course

on algorithms (principles, computational cost, scope, and limitations),

on (efficient and stable) implementation in C++ based on the numerical linear algebra EIGEN (a Domain

Specific Language embedded into C++)

on numerical experiments (design and interpretation).

(0.0.1) Aspects outside the scope of this course

No emphasis will be put on

theory and proofs (unless essential for understanding of algorithms). 401-3651-00L Numerical Methods for Elliptic and Parabolic Partial Differential Equations

401-3652-00L Numerical Methods for Hyperbolic Partial Differential Equations

(both courses offered in BSc Mathematics)

hardware aware implementation (cache hierarchies, CPU pipelining, etc.) 263-2300-00L How To Write Fast Numerical Code (Prof. M. Pschel, D-INFK)

issues of high-performance computing (HPC, shard and distributed memory parallelisation, vector-ization)

401-0686-10L High Performance Computing for Science and Engineering (HPCSE, Profs. M. Troyer

and P. Koumoutsakos)

263-2800-00L Design of Parallel and High-Performance Computing (Prof. T. Hfler)

However, note that these other courses partly rely on knowledge of elementary numerical methods, which

is covered in this course.

Contents

(0.0.2) Prequisites

8
http://eigen.tuxfamily.org


This course will take for granted basic knowledge of linear algebra, calculus, and programming, that you

should have acquired during your first year at ETH.

Analysis Linear algebra Programming (in C++)

Lin

ea

rsyste

ms

of

eq

ua

tio

ns

Eig

enva

lue

pro

ble

ms

Le

ast

sq

ua

res

pro

ble

ms

Inte

rpo

latio

n

Qu

ad

ratu

re

Nu

me

rica

l

inte

gra

tio

no

fO

DE

s

(0.0.3) Numerical methods: A motley toolbox

This course discusses elementary numerical methods and techniques

They are vastly different in terms of ideas, design, analysis, and scope of application. They are the

items in a toolbox, some only loosely related by the common purpose of being building blocks for

codes for numerical simulation.

Do not expect much coherence between the chapters of this course!

Fig. 1

A purpose-oriented notion of Numerical methods for

CSE:

A: Stop putting a hammer, a level, and duct tape

in one box! They have nothing to do with each

other!

B: I might need any of these tools when fixing some-

thing about the house

(0.0.4) Dependencies of topics

Despite the diverse nature of the individual topics covered in this course, some depend on others for

providing essential building blocks. The following directed graph tries to capture these relationships. The

arrows have to be read as uses results or algorithms of.

0. Introduction, 0. Introduction 9


Clustering techniques,

(G(xi , yj))i,jx, Ch. ??Numerical integration

y = f(t, y), Chapter 11

Quadraturef (x)dx,

Chapter 7

Eigenvalues

Ax = x, Chapter 9Krylov meth.,

Chapter 10

Function approximation,

Chapter 6

Least squares,

Ax b min,Chapter 3

Non-linear least squares,

F(x) min, Section 8.6

Interpolation

i ib(xi) = f (xi), Chapter 5Linear systems

Ax = b, Chapter 2

Non-linear systems

F(x)!= 0, Chapter 8

Filtering, Chapter 4 Sparse matrices, Section 2.7

Computing with matrices and vectors, Ch. 1 Zero finding f (x)!= 0

Any one-semester course Numerical methods for CSE will cover only selected chapters and sec-

tions of this document. Only topics addressed in class or in homework problems will be relevant

for the final exam!

(0.0.5) Relevance of this course

I am a student of computer science. After the exam, may I safely forget everything I have learned in this

mandatory numerical methods course? No, because it is highly likely that other courses or projects

will rely on the contents of this course:

singular value decomposition

least squares

}Computational statistics, machine learning

function approximation

numerical quadrature

numerical integration

Numerical methods for PDEs

interpolation

least squares

}Computer graphics

eigensolvers

sparse linear systems

}Graph theoretic algorithms

numerical integration}

Computer animation

and many more applications of fundamental numerical methods . . ..



Hardly anyone will need everything covered in this course, but most of you will need something.

0.0.2 Goals

Knowledge of the fundamental algorithms in numerical mathematics

Knowledge of the essential terms in numerical mathematics and the techniques used for the analysis

of numerical algorithms

Ability to choose the appropriate numerical method for concrete problems

Ability to interpret numerical results

Ability to implement numerical algorithms efficiently in C++ using numerical libraries

Indispensable: Learning by doing ( exercises)

0.0.3 To avoid misunderstandings . . .

(0.0.6) Lecture notes

These course materials are neither a textbook nor comprehensive lecture notes.

They are meant to be supplemented by explanations given in class.

Some pieces of advice:

the lecture material is not designed to be self-contained, but is to be studied beside attending the

course or watching the course videos,

this document is not meant for mere reading, but for working with,

turn pages all the time and follow the numerous cross-references,

study the relevant section of the course material when doing homework problems,

study referenced literature to refresh prerequisite knowledge and for alternative presentation of the

material (from a different angle, maybe), but be careful about not getting confused or distracted by

information overload.

(0.0.7) Comprehension is a process . . .

The course is difficult and demanding (ie. ETH level)

Do not expect to understand everything in class. The average student will

understand about one third of the material when attending the lectures,



understand another third when making a serious effort to solve the homework problems, hopefully understand the remaining third when studying for the examination after the end of

the course.

Perseverance will be rewarded!

0.0.4 Reporting errors

As the documents will always be in a state of flux, they will inevitably and invariably teem with small errors,

mainly typos and omissions.

Please report errors in the l,ecture material through the Course Wiki!

Join information will be sent to you by email.

Please point out errors by leaving a comment in the

Wiki (Discuss menu item).

When reporting an error, please specify the section and the number of the paragraph, remark, equation,

etc. where it hides. You need not give a page number.

0.0.5 Literature

Parts of the following textbooks may be used as supplementary reading for this course. References to

relevant sections will be provided in the course material.

Studying extra literature is not important for following this course!

http://numcseeth.wikispaces.com/


[?] U. ASCHER AND C. GREIF, A First Course in Numerical Methods, SIAM, Philadelphia, 2011.

Comprehensive introduction to numerical methods with an algorithmic focus based on MATLAB.

(Target audience: students of engineering subjects)

[3] W. DAHMEN AND A. REUSKEN, Numerik fr Ingenieure und Naturwissenschaftler, Springer, Hei-

delberg, 2006.

Good reference for large parts of this course; provides a lot of simple examples and lucid explana-

tions, but also rigorous mathematical treatment.

(Target audience: undergraduate students in science and engineering)

Available for download at PDF

[7] M. HANKE-BOURGEOIS, Grundlagen der Numerischen Mathematik und des Wissenschaftlichen

Rechnens, Mathematische Leitfden, B.G. Teubner, Stuttgart, 2002.

Gives detailed description and mathematical analysis of algorithms and relies on MATLAB. Profound

treatment of theory way beyond the scope of this course. (Target audience: undergraduates in

mathematics)

[?] A. QUARTERONI, R. SACCO, AND F. SALERI, Numerical mathematics, vol. 37 of Texts in Applied

Mathematics, Springer, New York, 2000.

Classical introductory numerical analysis text with many examples and detailed discussion of algo-

rithms. (Target audience: undergraduates in mathematics and engineering)

Can be obtained from website.

[5] P. DEUFLHARD AND A. HOHMANN, Numerische Mathematik. Eine algorithmisch orientierte Ein-

fhrung, DeGruyter, Berlin, 1 ed., 1991.

Modern discussion of numerical methods with profound treatment of theoretical aspects (Target

audience: undergraduate students in mathematics).

[?]: W.. GANDER, M.J. GANDER, AND F. KWOK, Scientific Computing, Text in Computational Sci-

ence and Engineering, springer, 2014.

Comprehensive treatment of elementary numerical methods with an algorithmic focus.

D-INFK maintains a webpage with links to some of these books.

Essential prerequisite for this course is a solid knowledge in linear algebra and calculus. Familiarity with

the topics covered in the first semester courses is taken for granted, see

[9] K. NIPP AND D. STOFFER, Lineare Algebra, vdf Hochschulverlag, Zrich, 5 ed., 2002.

[?] M. GUTKNECHT, Lineare algebra, lecture notes, SAM, ETH Zrich, 2009, available online.

[12] M. STRUWE, Analysis fr Informatiker. Lecture notes, ETH Zrich, 2009, available online.

https://books.google.ch/books/about/A_First_Course_on_Numerical_Methods.html?id=gJjh6QcBrlEC&redir_esc=yhttp://books.google.de/books?id=zWenT-hxDxEChttp://link.springer.com/book/10.1007%2F978-3-540-76493-9http://books.google.de/books?id=NxzXX04UBIYChttp://books.google.de/books?id=31m4ahn_KfkChttp://link.springer.com/book/10.1007%2Fb98885http://books.google.de/books?id=1oDXWLb9qEkChttp://www.springer.com/de/book/9783319043241http://textbooks.inf.ethz.ch/lectures/compulsory-courses/7/http://books.google.de/books?id=BvcHtRVy_N8Chttp://www.sam.math.ethz.ch/~mhg/unt/LA/HS07/http://www.math.ethz.ch/~struwe/Skripten/InfAnalysis-I-II-31-7-09.pdf


0.1 Specific information

0.1.1 Assistants and exercise classes

Lecturer: Prof. Ralf Hiptmair HG G 58.2, 044 632 3404, [email protected]

Assistants: Daniele Casati, HG E 62.2, 044 632 ????, [email protected]

Filippo Leonardi, HG J 45, 044 633 9379, [email protected]

Daniel Hupp, [email protected]

Marija Kranjcevic, [email protected]

Heinekamp Sebastian, [email protected]

Hillebrand Fabian, [email protected]

Schaffner Yannick, [email protected]

Thomas Graf, [email protected]

Schwarz Fabian, [email protected]

Accaputo Giuseppe, [email protected]

Baumann Christian, [email protected]

Romero Francisco, [email protected]

Dabrowski Alexander, [email protected]

Luca Mondada, [email protected]

Varghese Alexander , [email protected]

Xandeep,

Though the assistants email addresses are provided above, their use should be restricted to cases of

emergency:

In general refrain from sending email messages to the lecturer or the assistants. They will not

be answered!

Questions should be asked in class (in public or during the break in private), in the tutorials, or

in the study center hours.

Classes: Thu, 08.15-10.00 (HG F 1), Fri, 13.15-16.00 (HG F 1)

Tutorials: Mon, 10.15-12.00 (CLA E 4, LFW E 11, LFW E13, ML H 41.1, ML J 34.1)

Mon, 13.15-15.00 (CLA E 4, HG E 33.3, HG E 33.5, HG G 26.5, LEE D 105)

Study center: Mon, 18.00-20.00 (HG E 41)

Before the first tutorial you will receive a link where you can register to a tutorial class. Keep in mind that

one tutorial will be held in German and one will be reserved for CSE students.

0. Introduction, 0.1. Specific information 14
http://www.sam.math.ethz.ch/~hiptmair


0.1.2 Study center

The tutorials classes for this course will be supple-

mented by the option to do supervised work in the

ETH flexible lecture hall (study center) HG E 41 .

Several assistants will be present to explain and dis-

cuss homework problems both from the previous and

the current problem sheet. They are also supposed

to answer questions about the course.

The study center session is also a good opportunity

to do the homework in a group. In case you are

stalled you may call an assistant and ask for advice.Fig. 2

0.1.3 Assignments

A steady and persistent effort spent on homework problems is essential for success in this course.

You should expect to spend 4-6 hours per week on trying to solve the homework problems. Since many

involve small coding projects, the time it will take an individual student to arrive at a solution is hard to

predict.

(0.1.1) Homeworks and tutors corrections

The weekly assignments will be a few problems from the NCSE Problem Collection available online

as PDF. The particular problems to be solved will be communicated on Friday every week.

Please note that this problem collection is being compiled during this semester. Thus, make sure

that you obtain the most current version every week.

Some or all of the problems of an assignment sheet will be discussed in the tutorial classes on

Monday 10 days after the problems have been assigned.

A few problems on each sheet will be marked as core problems. Every participant of the course is

strongly advised to try and solve at least the core problems.

If you want your tutor to examine your solution of the current problem sheet, please put it into the

plexiglass trays in front of HG G 53/54 by the Thursday after the publication. You should submit your

codes using the online submission interface. This is voluntary, but feedback on your performance

on homework problems can be important.

Please clearly mark the homework problems that you want your tutor to inspect.

You are encouraged to hand-in incomplete and wrong solutions, you can receive valuable feedback

even on incomplete attempts.

(0.1.2) Git code repository

http://www.sam.math.ethz.ch/~grsam/NumCSE/NumCSEProblems16.pdfhttps://people.math.ethz.ch/~grsam/submit/


C++ codes for both the classroom and homework problems are made available through a git repository

also accessible through Gitlab (Link):

The Gitlab toplevel page gives a short introduction into the repository for the course and provides a link to

online sources of information about Git.

Ddownload is possible via Git or as a zip archive. Which method you choose is up to you, but it should be

noted that updating via git is more convenient.

Shell command to download the git repository:

> git clone https://gitlab.math.ethz.ch/NumCSE/NumCSE.git

Updating the repository to fetch upstream changes is then possible by executing > git pull inside the

NumCSE folder.

Note that by default participants of the course will have read access only. However, if you want to contribute

corrections and enhancements of lecture or homework codes your are invited to submit a merge request.

Beforehand you have to inform your tutor so that a personal Gitlab account can be set up for you.

The Zip-archive download link is here.

For instructions on how to compile assignments or lecture codes see the README file.

0.1.4 Information on examinations

(0.1.3) Examinations during the teaching period

From the ETH course directory:

Computer based examination involving coding problems beside theoretical questions. Parts

of the lecture documents and other materials will be made available online during the exami-

nation. A 30-minute mid-term exam and a 30-minute end term exam will be held during the

teaching period on dates specified in the beginning of the semester. Points earned in these

exams will be taken into account through a bonus of up to 20% of the total points in the final

session exam.

Both will be closed book examinations on paper.

Dates:

https://gitlab.math.ethz.ch/NumCSE/NumCSEhttps://gitlab.math.ethz.ch/NumCSE/NumCSE/repository/archive.zip?ref=masterhttps://gitlab.math.ethz.ch/NumCSE/NumCSE/blob/master/README.md


Mid-term: Friday, Nov 4, 2016, 13:15 End-term: Friday, Dec 23, 2016, 13:15 Make-up term exam: Friday, Jan 13, 2017, 9:15 Repetition term exam: Friday, May 12, 2017, 16:15

The term exams are regarded as central elements and as such are graded on a pass/fail basis.

Admission to the main exam is conditional on passing at least one term exam

Only students who could not take part in one of the term exams for cogent reasons like illness (doc-

tors certificate required!) may take part in the make-up term exam. Please contact Daniele Casati

([email protected]) by email, if you think that you are eligible for the make-up term exam,

and attach all required documentation. You will be informed, whether you are admitted.

Only students who have failed both term exams can take part in the repetition term exam in spring next

year. This is their only chance to be admitted to the main exam in Summer 2017.

(0.1.4) Main examination during exam session

Three-hour written examination involving coding problems to be done at the computer on

Thursday January 26, 2017, 9:00 - 12:00, HG G 1

Dry-run for computer based examination:

TBA, registration via course website

Subjects of examination:

All topics, which have been addressed in class or in a homework problem (including the home-work problems not labelled as core problems)

Lecture documents will be available as PDF during the examination. The corresponding final version

of the lecture documents will be made available on TBA

You may bring a summary of up to 10 pages A4 in your own handwriting. No printouts and copies

are allowed.

The exam questions will be asked in English.

(0.1.5) Repeating the main exam

Everybody who passed at least one of the term exams, the make-up term exam, or the repetitionterm exam for last years course and wants to repeat the main exam, will be allowed to do so.

Bonus points earned in term exams in last years course can be taken into account for this coursesmain exam.



If you are going to repeat the main exam, but also want to earn a bonus through this years termexams, please declare this intention before the mid-term exam.


Bibliography

[1] S. Brm and M. Melenk. Approximation of the high-frequency helmholtz kernel by nested directional

interpolation. Preprint arXiv:1510.07189 [math.NA], arXiv, 2015.

[2] Q. Chen and I. Babuska. Approximate optimal points for polynomial interpolation of real functions in

an interval and in a triangle. Comp. Meth. Appl. Mech. Engr., 128:405417, 1995.

[3] W. Dahmen and A. Reusken. Numerik fr Ingenieure und Naturwissenschaftler. Springer, Heidelberg,

2008.

[4] P.J. Davis. Interpolation and Approximation. Dover, New York, 1975.

[5] P. Deuflhard and A. Hohmann. Numerical Analysis in Modern Scientific Computing, volume 43 of

Texts in Applied Mathematics. Springer, 2003.

[6] C.A. Hall and W.W. Meyer. Optimal error bounds for cubic spline interpolation. J. Approx. Theory,

16:105122, 1976.

[7] M. Hanke-Bourgeois. Grundlagen der Numerischen Mathematik und des Wissenschaftlichen Rech-

nens. Mathematische Leitfden. B.G. Teubner, Stuttgart, 2002.

[8] Matthias Messner, Martin Schanz, and Eric Darve. Fast directional multilevel summation for oscilla-

tory kernels based on Chebyshev interpolation. J. Comput. Phys., 231(4):11751196, 2012.

[9] K. Nipp and D. Stoffer. Lineare Algebra. vdf Hochschulverlag, Zrich, 5 edition, 2002.

[10] R. Rannacher. Einfhrung in die numerische mathematik. Vorlesungsskriptum Universitt Heidel-

berg, 2000. http://gaia.iwr.uni-heidelberg.de/.

[11] R. Remmert. Funktionentheorie I. Number 5 in Grundwissen Mathematik. Springer, Berlin, 1984.

[12] M. Struwe. Analysis fr Informatiker. Lecture notes, ETH Zrich, 2009. https://moodle-

app1.net.ethz.ch/lms/mod/resource/index.php?id=145.

[13] P. Vertesi. On the optimal lebesgue constants for polynomial interpolation. Acta Math. Hungaria,

47(1-2):165178, 1986.

[14] P. Vertesi. Optimal lebesgue constant for lagrange interpolation. SIAM J. Numer. Aanal., 27(5):1322

1331, 1990.

0.2 Programming in C++11

C++11 is the current ANSI/ISO standard for the programming language C++. On the one hand, it offers

a wealth of features and possibilities. On the other hand, this can be confusing and even be prone to

19


inconsistencies. A major cause of inconsistent design is the requirement with backward compatibility with

the C programming language and the earlier standard C++98.

However, C++ has become the main language in computational science and engineering and high per-

formance computing. Therefore this course relies on C++ to discuss the implementation of numerical

methods.

In fact C++ is a blend of different programming paradigms:

an object oriented core providing classes, inheritance, and runtime polymorphism, a powerful template mechanism for parametric types and partial specialization, enabling template

meta-programming and compile-time polymorphism,

a collection of abstract data containers and basic algorithms provided by the Standard TemplateLibary (STL).

Supplementary reading. A popular book for learning C++ that has been upgraded to include

the latest C++11 standard is [?].

The book [?] gives a comprehensive presentation of the new features of C++11 compared to earlier

versions of C++.

There are plenty of online reference pages for C++, for instance http://en.cppreference.com

and http://www.cplusplus.com/.

(0.2.1) Building, compiling, and debugging

We use the command line build tool CMAKE, see web page. The compilers supporting all features of C++ needed for this course, are clang and GCC. Both are

open source projects and free. CMAKE will automatically select a suitable compiler on your system

(Linux or Mac OS X).

A command line tool for debugging is lldb, see short introduction by Till Ehrengruber, student ofCSE@ETH.

The following sections highlight a few particular aspects of C++11 that may be important for code devel-

opment in this course.

0.2.1 Function Arguments and Overloading

(0.2.2) Function overloading

Argument types are an integral part of a function declaration in C++. Hence the following functions are

different

i n t* f( i n t); // use this in the case of a single numeric argument

double f( i n t *); // use only, if pointer to a integer is given

void f(const MyClass &); // use when called for a MyClass object

0. Introduction, 0.2. Programming in C++11 20
http://en.cppreference.comhttp://www.cplusplus.com/https://cmake.org/http://clang.llvm.org/https://gcc.gnu.org/https://gitlab.math.ethz.ch/tille/debugging-cpp-code-with-lldb


and the compiler selects the function to be used depending on the type of the arguments following rather

sophisticated rules, refer to overload resolution rules. Complications arise, because implicit type conver-

sions have to be taken into account. In case of ambiguity a compile-time error will be triggered. Functions

cannot be distinguished by return type!

For member functions (methods) of classes an additional distinction can be introduced by the const spec-

ifier:

s t r u c t MyClass {

double f(double); // use for a mutable object of type MyClass

double f(double) const; // use this version for a constant object

...

};

The second version of the method f is invoked for constant objects of type MyClass.

(0.2.3) Operator overloading

In C++ unary and binary operators like =, ==, +, -, *, /, +=, -=, *=, /=, %, &&, ||, etc. are regarded

as functions with a fixed number of arguments (one or two). For built-in numeric and logic types they are

defined already. They can be extended to any other type, for instance

MyClass opera tor +(const MyClass &,const MyClass &);

MyClass opera tor +(const MyClass &,double);

MyClass opera tor +(const MyClass &); // unary + !

The same selection rules as for function overloading apply. Of course, operators can also be introduced

as class member functions.

C++ gives complete freedom to overload operators. However, the semantics of the new operators should

be close to the customary use of the operator.

(0.2.4) Passing arguments by value and by reference

Consider a generic function declared as follows:

void f(MyClass x); // Argument x passed by value.

When f is invoked, a temporary copy of the argument is created through the copy constructor or the move

constructor of MyClass. The new temporary object is a local variable inside the function body.

When a function is declared as follows

void f(MyClass &x); // Argument x passed by reference.

then the argument is passed to the scope of the function and can be changed inside the function. No

copies are created. If one wants to avoid the creation of temporary objects, which may be costly, but also

wants to indicate that the argument will not be modified inside f, then the declaration should read

void f(const MyClass &x); // Argument x passed by constant referene.

http://en.cppreference.com/w/cpp/language/overload_resolution.htmlhttp://en.cppreference.com/w/cpp/language/operators.htmlhttp://en.cppreference.com/w/cpp/language/expressions.html#Operators


New in C++11 is move semantics, enabled in the following definition

void f(const MyClass &&x); // Optional shallow copy

In this case, if the scope of the object passed as the argument is merely the function or std::move()

tags it as disposable, the move constructor of MyClass is invoked, which will usually do a shallow copy

only. Refer to Code 0.2.22 for an example.

0.2.2 Templates

(0.2.5) Function templates

The template mechanism supports parameterization of definitions of classes and functions by type. An

example of a function templates is

template

VectorType saxpy(ScalarType alpha,const VectorType &x,const

VectorType &y)

{ r e t u r n(alpha*x+y); }

Depending on the concrete type of the arguments the compiler will instantiate particular versions of this

function, for instance saxpy, when alpha is of type float and both x and y are of

type double. In this case the return type will be float.

For the above example the compiler will be able to deduce the types ScalarType and VectorType

from the arguments. The programmer can also specify the types directly through the < >-syntax as in

saxpy(a,x,y);

If an instantiation for all arguments of type double is desired. In case, the arguments do not supply

enough information about the type parameters, specifying (some of) them through < > is mandatory.

(0.2.6) Class templates

A class template defines a class depending on one or more type parameters, for instance

template

c lass MyClsTempl {

p u b l ic:

using parm_t = typename T::value_t; // T-dependent type

MyClsTempl(void); // Default constructor

MyClsTempl(const T&); // Constructor with an argument

template

T memfn(const T&,const U&) const; // Templated member function

p r i v a t e:

T *ptr; // Data member, T-pointer

};



Types MyClsTempl for a concrete choice of T are instantiated when a corresponding object is de-

clared, for instance via

double x = 3.14;

MyClass myobj; // Default construction of an object

MyClsTempl tinstd; // Instantiation for T = double

MyClsTempl mytinst(myobj); // Instantiation for T = MyClass

MyClass ret = mytinst.memfn(myobj,x); // Instantiation of member

function for U = double, automatic type deduction

The types spawned by a template for different parameter types have nothing to do with each other.

Requirements on parameter types

The parameter types for a template have to provide all type definitions, member functions, operators,

and data to make possible the instantiation (compilation) of the class of function template.

0.2.3 Function Objects and Lambda Functions

A function object is an object of a type that provides an overloaded function call operator (). Function

objects can be implemented in two different ways:

(I) through special classes like the following that realizes a function R 7 Rc lass MyFun {

p u b l ic:

...

double operator (double x) const; // Evaluation operator

...

};

The evaluation operator can take more than one argument and need not be declared const.

(II) through lambda functions, an anonymous function defined as

[] () -> { body; }

where is a list of variables from the local scope to be passed to the lambda func-

tion; an & indicates passing by reference,

is a comma separated list of function arguments complete with types,

is an optional return type; often the compiler will be able to deduce the

return type from the definition of the function.

Function classes should be used, when the function is needed in different places, whereas lambda func-

tions for short functions intended for single use.

C++11 code 0.2.8: Demonstration of use of lambda function

1 i n t main ( ) {

2 // initialize a vector from an initializer list

3 std : : vector v ( { 1 . 2 , 2 . 3 , 3 . 4 , 4 . 5 , 5 . 6 , 6 . 7 , 7 . 8 } ) ;



4 // A vector of the same length

5 std : : vector w( v . size ( ) ) ;

6 // Do cumulative summation of v and store result in w

7 double sum = 0;

8 std : : transform ( v . begin ( ) , v . end ( ) ,w. begin ( ) ,

9 [&sum ] ( double x ) { sum += x ; return sum ; } ) ;

10 cout


C++11 code 0.2.11: Function with multiple return values

1 template

2 std : : tuple extcumsum( const std : : vector &v ) {

3 // Local summation variable captured by reference by lambda function

4 T sum { } ;

5 // temporary vector for returning cumulative sum

6 std : : vector w { } ;

7 // cumulative summation

8 std : : t rans form ( v . cbegin ( ) , v . cend ( ) , back_ inse r te r (w) ,

9 [&sum ] (T x ) { sum += x ; return (sum) ; } ) ;

10 return ( std : : tuple

11 ( std : : min_element ( v . cbegin ( ) , v . cend ( ) ) ,12 std : : max_element ( v . cbegin ( ) , v . cend ( ) ) ,13 std : : move(w) ) ) ;

14 }

This code snipped shows how to extract the individual components of the tuple returned by the previous

function.

C++11 code 0.2.12: Calling a function with multiple return values

1 i n t main ( ) {

2 // initialize a vector from an initializer list

3 std : : vector v ( { 1 . 2 , 2 . 3 , 3 . 4 , 4 . 5 , 5 . 6 , 6 . 7 , 7 . 8 } ) ;

4 // Variables for return values

5 double minv , maxv ; // Extremal elements

6 std : : vector cs ; // Cumulative sums

7 std : : t i e ( minv , maxv , cs ) = extcumsum( v ) ;

8 cout


4 using va lue_t = double ;

5 // Constructor creating constant vector, also default constructor

6 e x p l i c i t MyVector ( std : : size_t n = 0 ,double va l = 0 .0 ) ;

7 // Constructor: initialization from an STL container

8 template MyVector ( const Conta iner &v ) ;

9 // Constructor: initialization from an STL iterator range

10 template MyVector ( I t e r a t o r f i r s t , I t e r a t o r l a s t ) ;

11 // Copy constructor, computational cost O(n)

12 MyVector ( const MyVector &mv) ;

13 // Move constructor, computational cost O(1)

14 MyVector ( MyVector &&mv) ;

15 // Copy assignment operator, computational cost O(n)

16 MyVector &operator = ( const MyVector &mv) ;

17 // Move assignment operator, computational cost O(1)

18 MyVector &operator = ( MyVector &&mv) ;

19 // Destructor

20 v i r t u a l ~MyVector ( void ) ;

21 // Type conversion to STL vector

22 operator std : : vector ( ) const ;

23

24 // Returns length of vector

25 std : : size_t leng th ( void ) const { return n ; }

26 // Access operators: rvalue & lvalue, with range check

27 double operator [ ] ( std : : size_t i ) const ;

28 double &operator [ ] ( std : : size_t i ) ;

29 // Comparison operators

30 bool operator == ( const MyVector &mv) const ;

31 bool operator != ( const MyVector &mv) const ;

32 // Transformation of a vector by a function R R33 template

34 MyVector &trans form ( Functor &&f ) ;

35

36 // Overloaded arithmetic operators

37 // In place vector addition: x += y;

38 MyVector &operator +=(const MyVector &mv) ;

39 // In place vector subtraction: x-= y;

40 MyVector &operator =(const MyVector &mv) ;41 // In place scalar multiplication: x *= a;

42 MyVector &operator =(double alpha ) ;43 // In place scalar division: x /= a;

44 MyVector &operator / = ( double alpha ) ;

45 // Vector addition

46 MyVector operator + ( MyVector mv) const ;

47 // Vector subtraction

48 MyVector operator ( const MyVector &mv) const ;49 // Scalar multiplication from right and left: x = a*y; x = y*a

50 MyVector operator ( double alpha ) const ;51 f r iend MyVector operator ( double alpha , const MyVector &) ;52 // Scalar divsion: x = y/a;

53 MyVector operator / ( double alpha ) const ;



54

55 // Euclidean norm

56 double norm ( void ) const ;

57 // Euclidean inner product

58 double operator ( const MyVector &) const ;59 // Output operator

60 f r iend std : : ostream &

61 operator


C++11 code 0.2.18: Templated constructors copying vector entries from an STL container

1 template

2 MyVector : : MyVector ( const Conta iner &v ) : n ( v . size ( ) ) , data ( nul lp t r ) {

3 i f ( dbg ) cout


{ Des t ruc to r f o r MyVector ( leng th = 7 ) }



The copy constructor listed next relies on the STL algorithm std::copy to copy the elements of an

existing object into a newly created object. This takes n operations.

C++11 code 0.2.21: Copy constructor

1 MyVector : : MyVector ( const MyVector &mv) : n (mv. n ) , data ( nul lp t r ) {

2 i f ( dbg ) cout


{ MyVector ( leng th 8) cons t ruc ted from con ta ine r }

{ opera tor a , MyVector o f leng th 8}{Copy c o n s t r u c t i o n o f MyVector ( leng th 8 ) }

{ opera tor = , MyVector o f leng th 8}{Move c o n s t r u c t i o n o f MyVector ( leng th 8 ) }


{Move c o n s t r u c t i o n o f MyVector ( leng th 8 ) }

v1 = [ ]

v2 = [ 2 .4 ,4 .6 ,6 .8 ,9 ,11 .2 ,13 .4 ,15 .6 ,17 .8 ]

v3 = [ 1 .2 ,2 .3 ,3 .4 ,4 .5 ,5 .6 ,6 .7 ,7 .8 ,8 .9 ]




We observe that the object v1 is reset after having been moved to v3.

!Use std::move only for special purposes like above and only if an object has a move con-

structor. Otherwise a move will trigger a plain copy operation. In particular, do not use

std::move on objects at the end of their scope, e.g., within return statements.

The next operator effects copy assignment of an rvalue MyVector object to an lvalue MyVector. This

involves O(n) operations.

C++11 code 0.2.24: Copy assignment operator

1 MyVector &MyVector : : operator = ( const MyVector &mv) {

2 i f ( dbg ) cout


C++11 code 0.2.26: Destructor: releases allocated memory

1 MyVector : : ~ MyVector ( void ) {

2 i f ( dbg ) cout


11

12 bool MyVector : : operator != ( const MyVector &mv) const {

13 return ! ( th is == mv) ;14 }

The transform method applies a function to every vector component and overwrites it with the value

returned by the function. The function is passed as an object of a type providing a ()-operator that accepts

a single argument convertible to double and returns a value convertible to double.

C++11 code 0.2.30: Transformation of a vector through a functor double double1 template

2 MyVector &MyVector : : t rans form ( Functor &&f ) {

3 for ( std : : size_t l =0; l


The output is

8 operat ions , mv transformed = [ 3 .2 ,4 .3 ,5 .4 ,6 .5 ,7 .6 ,8 .7 ,9 .8 ,1 0 .9 ]

8 operat ions , mv transformed = [ 5 .2 ,6 .3 ,7 .4 ,8 .5 ,9 .6 ,10 .7 ,11 .8 ,12 .9 ]

F i n a l vec tor = [ 1 .2 ,2 .3 ,3 .4 ,4 .5 ,5 .6 ,6 .7 ,7 .8 ,8 .9 ]

Operator overloading provides the natural vector operations in Rn both in place and with a new vector

created for the result.

C++11 code 0.2.33: In place arithmetic operations (one argumnt)

1 MyVector &MyVector : : operator +=(const MyVector &mv) {

2 i f ( dbg ) cout


9 MyVector MyVector : : operator ( const MyVector &mv) const {10 i f ( dbg ) cout


4 double s = 0;

5 for ( std : : size_t l =0; l


{ opera tor + , MyVector o f leng th 8}

{Copy c o n s t r u c t i o n o f MyVector ( leng th 8 ) }

{ opera tor =, MyVector o f leng th 8}{ norm : MyVector o f leng th 8}

{ opera tor / , MyVector o f leng th 8}

{Copy c o n s t r u c t i o n o f MyVector ( leng th 8 ) }

{ opera tor = , MyVector o f leng th 8}{ opera tor + , MyVector o f leng th 8}

{ opera tor += , MyVector o f leng th 8}

{Move c o n s t r u c t i o n o f MyVector ( leng th 8 ) }









Several temporary objects are created and destroyed and quite a few copy operations take place. The

situation would be worse unless move semantics was available; if we had not supplied a move constructor,

a few more copy operations would have been triggered. Even worse, the frequent copying of data runs a

high risk of cache misses. This is certainly not an efficient way to do elementary vector operations though

it looks elegant at first glance.

Example 0.2.41 (Gram-Schmidt orthonormalization based on MyVector implementation)

Gram-Schmidt orthonormalization has been taught in linear algebra and its theory will be revisited in

1.5.1. Here we use this simple algorithm from linear algebra to demonstrate the use of the vector class

MyVector defined in Code 0.2.13.

The templated function gramschmidt takes a sequence of vectors stored in a std::vector object. The

actual vector type is passed as a template parameter. It has to supply length and norm member

functions as well as in place arithmetic operations -=, / and =. Note the use of the highlighted methods

of the std::vector class.

C++11 code 0.2.42: templated function for Gram-Schmidt orthonormalization

1 template

2 std : : vector gramschmidt ( const std : : vector &A , double

eps=1E14) {3 const in t k = A . size ( ) ; // no. of vectors to orthogonalize

4 const in t n = A [ 0 ] . leng th ( ) ; // length of vectors

5 cout


11 Q. pop_back ( ) ; break ;

12 }

13 Q. back ( ) /= Q. back ( ) .norm ( ) ; // normalization

14 }

15 return (Q) ; // return at end of local scope

16 }

This driver program calls a function that initializes a sequence of vectors and then orthonormalizes them

by means of the Gram-Schmidt algorithm. Eventually orthonormality of the computed vectors is tested.

Please pay attention to

the use of auto to avoid cumbersome type declarations, the for loops following the foreach syntax. automatic indirect template type deduction for the templated function gramschmidt from its argu-

ment. In Line 6 the function gramschmidt is instantiated.

C++11 code 0.2.43: Driver code for Gram-Schmidt orthonormalization

1 i n t main ( ) {

2 myvec : : MyVector : : dbg = fa lse ;

3 const in t n = 7; const in t k = 7;

4 auto A( in i t vec tors ( n , k , [ ] ( i n t i , i n t j )

5 { return std : : min ( i +1 , j +1) ; } ) ) ;

6 auto Q( gramschmidt (A) ) ; // instantiate template for MyVector

7 cout


9 }

10 return (A) ;

11 }

0.3 Creating Plots with MATHGL

0.3.1 MATHGL Documentation (by J. Gacon)

MATHGL is a huge open-source plotting library for scientific graphics. It can be used for many programming

languages, in particular also for C++. Mainly we will use the Figure library (implemented by J. Gacon,

student of CSE@ETH) introduced below (Section 0.3.4).

However for some special plots using MATHGL can be necessary. A full documentation can be found at

http://mathgl.sourceforge.net/doc_en/index.html.

First of all note that all MATHGL plot commands do not take std::vectors or EIGEN vectors as argu-

ments but only mglData.

NOTE: The Figure environment takes care of all this conversion and formatting!

From std::vector or Eigen::VectorXd a mglData can be initialized as follows:

C++ code 0.3.1: Vector to mglData

1 std : : vector v ;

2 Eigen : : VectorXd w;

3 // ... initialize data...

4

5 // the constructor takes a pointer to the first element

6 // and the size of the vector

7 mglData vd ( v . data ( ) , v . size ( ) ) ,

8 wd(w. data ( ) , w. size ( ) ) ;

For Eigen::RowVectorXd we first must do some rearrangements of the vector, as the data()

method returns a pointer to the column major data.

C++ code 0.3.2: Eigen::RowVectorXd to mglData

1 Eigen : : RowVectorXd v ;

2 // ... initialize data...

3

4 // Store data in a temporary array

5 std : : vector tmp ( v . cols ( ) ) ;

6 for ( long i = 0 ; i < r . cols ( ) ; ++ i ) tmp [ i ] = v ( i ) ;

7 mglData vd ( tmp . data ( ) , tmp . size ( ) ) ;

For matrices the constructor is slightly different:

0. Introduction, 0.3. Creating Plots with MATHGL 38
http://mathgl.sourceforge.net/doc_en/index.html


C++ code 0.3.3: Matrices to mglData

1 Eigen : : MatrixXd A;

2 // ... initialize data ...

3

4 // The column major data layout fits MATHGL conventions

5 mglData Ad(A . rows ( ) , A . cols ( ) , A . data ( ) ) ;

0.3.2 MATHGL Installation

If youre using Linux the easiest way to install MATHGL is via the command line: apt-get install

mathgl (Ubuntu/Debian) or dnf install mathgl (Fedora).

You can also use CMake to install it.

1. Install CMake (apt-get install cmake or from https://cmake.org/download/)

2. Download the MATHGL source from

http://mathgl.sourceforge.net/doc_en/Download.html#Download

3. In the mathgl-2.3.* directory do:

mkdir build && cd build

cmake ..

make (this step might take a while)

(sudo) make install

0.3.3 Corresponding Plotting functions of MATLAB and MATHGL

MATHGL offers a subset of MATLABs large array of plotting functions. The following tables list correspond-

ing commands/methods:

Plotting in 1-D

https://cmake.org/download/http://mathgl.sourceforge.net/doc_en/Download.html#Download


MATLAB MATHGL

axis([0,5,-2,2]) gr.Ranges(0,5,-2,2)

Default: autofit (axis auto) Default: x=-1:1, y=-1:1

Workaround:

gr.Ranges(x.Minimal(), x.Maximal(),

y.Minimal(), y.Maximal())

axis([0,5,-inf,inf]) gr.Range(x,0,5)

axis([-inf,inf,-2,2]) gr.Range(y,-2,2)

xlabel(x-axis) gr.Label(x, "x-axis")

ylabel(y-axis) gr.Label(y, "y-axis")

legend(sin(x), x2) gr.AddLegend("sin(x)","b")

gr.AddLegend("\\x2", "g")

gr.Legend()

legend(exp(x)) gr.AddLegend("exp(x)","b")

legend(boxoff) gr.Legend(1,1,"")

legend(x,Location, gr.AddLegend("x","b")

northwest) gr.Legend(0,1)

legend(cos(x), gr.AddLegend("cos(x)","b")

Orientation,horizontal) gr.Legend("#-")

(0, 1) (0.5, 1) (1, 1)(0, 0.5) (0.5, 0.5) (1, 0.5)(0, 0) (0.5, 0) (1, 0)

Legend alignment in MathGL:

Values larger than 1 will give position outside of the graph.

Default is (1,1).

plot(y) gr.Plot(y)

plot(t,y) gr.Plot(t,y)

plot(t0,y0,t1,y1) gr.Plot(t0,y0)

gr.Plot(t1,y1)

plot(t,y,b+) gr.Plot(t,y,"b+")

print(myfig,-depsc) gr.WriteEPS("myfig.eps")

print(myfig,-dpng) gr.WritePNG("myfig.png")

(compile w/ flag -lpng)

title(Plot title) gr.Title("Plot title") (title high above plot)

gr.Subplot(1,1,0,"


0.3.4 The Figure Class

The Figure library is an interface to MATHGL. By taking care of formatting and the layout it allows a very

simple, fast and easy use of the powerful plot library.

This library depends on MATHGL (and optionally on EIGEN), so the installation requires a working version

of these dependencies.

0.3.4.1 Introductory example

This short example code will show, how the Figure class can be used.

C++11 code 0.3.4: [

A first code using MATHGL ]

1 # include

2 # include

3 # include < f i g u r e / f i g u r e . hpp>

4

5 i n t main ( ) {

6 std : : vector x (10) , y (10) ;

7 for ( i n t i = 0 ; i < 10; ++ i ) {

8 x [ i ] = i ; y [ i ] = std : : exp(0.2 i ) std : : cos ( i ) ;9 }

10 Eigen : : VectorXd u = Eigen : : VectorXd : : LinSpaced (500 , 0 , 9) ,

11 v = ( u . array ( ) . cos ( ) (0.2u ) . array ( ) . exp ( )) . matrix ( ) ;

12 mgl : : Figure f i g ;

13 f i g . plot ( x , y , " + r " ) . label ( " Sample Data " ) ;

14 f i g . plot ( u , v , " b " ) . label ( " F u n c t i o n " ) ;

15 f i g . legend ( ) ;

16 f i g . save ( " p l o t . eps " ) ;

17 return 0;

18 }



The figure beside displays the graphical output

stored in EPS format in the file plot.eps.

Note the use of the methods label and legend to

create the legend.

Fig. 3

0.3.4.2 Figure Methods

(0.3.5) The grid method

This method prints gridlines in the background of a plot.

Definition:

void grid( const bool& on = t rue ,

const std::string& gridType = "-",

const std::string& gridCol = "h" )

Restrictions: None.

Examples:

1 mgl : : F igure f i g ;

2 f i g . p l o t ( x , y ) ;

3 f i g . grid ( ) ; // set grid

4 f i g . save ( " p l o t . eps " ) ;

5


7 f i g . p l o t ( x , y ) ;

8 f i g . grid ( true , " ! " , " h " ) ; // grey (-> h) fine (-> !) mesh

9 f i g . save ( " p l o t . eps " ) ;



Fig. 4 Fig. 5

(0.3.6) The xlabel method

This method adds a label text to the x-axis.

Definition:

void xlabel( const std::string& label,

const double& pos = 0 )

Restrictions: None.

Examples:


2 f i g . p l o t ( x , y , " g+ " ) ; // g+ equals matlab +-g

3 f i g . xlabel ( " L i n e a r x a x i s " ) ;

4 f i g . save ( " p l o t . eps " ) ;

5


7 f i g . xlabel ( " L o g a r i t h m i x x a x i s " ) ; // no restricitons on call order

8 f i g . se t log ( true , true ) ;

9 f i g . p l o t ( x , y , " g+ " ) ;

10 f i g . save ( " p l o t . eps " ) ;



Fig. 6 Fig. 7

(0.3.7) The ylabel method

This command adds text to the y-axis.

Definition:

void ylabel( const std::string& label,

const double& pos = 0 )

Restrictions: None.

Examples: See xlabel.

(0.3.8) The legend method

This method adds a legend to a plot. Legend entries have to be defined by the label method given after

the plot command.

Definition:

void legend( const double& xPos = 1,

const double& yPos = 1 )

Restrictions: None.

Examples:


2 f i g . p l o t ( x0 , y0 ) . label ( " My F u n c t i o n " ) ;

3 f i g . legend ( ) ; // activate legend

4 f i g . save ( " p l o t " ) ;

5





8 f i g . legend ( 0 .5 , 0 .25) ; // set position to (0.5, 0.25)

9 f i g . save ( " p l o t " ) ;

10



13 f i g . save ( " p l o t " ) ; // legend wont appear as legend() hasnt been called

Fig. 8 Fig. 9

(0.3.9) The setlog method

Selecting axis scaling for a plot, either linear or logarithmic.

Definition:

void setlog( const bool& logx = f a ls e ,

const bool& logy = f a ls e ,

const bool& logz = f a l s e )

Restrictions: All plots will use the latest setlog options or default if none have been set.

Examples:


2 f i g . setlog ( true , fa lse ) ; // -> semilogx

3 f i g . p l o t ( x0 , y0 ) ;

4 f i g . setlog ( false , true ) ; // -> semilogy

5 f i g . p l o t ( x1 , y1 ) ;

6 f i g . setlog ( true , true ) ; // -> loglog

7 f i g . p l o t ( x2 , y2 ) ;

8 f i g . save ( " p l o t . eps " ) ; // ATTENTION: all plots will use loglog-scale

9




11 f i g . p l o t ( x , y ) ;

12 f i g . save ( " p l o t . eps " ) ; // -> default (= linear) scaling

Fig. 10 Fig. 11

(0.3.10) The plot method

The standard method for plotting function graphs.

Definition:

template

void plot( const yVector& y,

const std::string& style = "" )

template

void plot( const xVector& x,

const yVector& y,


Restrictions: xVector and yVector must have a size() method, which returns the size of the vec-

tor and a data() method, which returns a pointer to the first element in the vector.

Furthermore x and y must have same length.

Examples:


2 f i g . plot ( x , y , " g ; " ) ; // green and dashed linestyle

3 f i g . save ( " da ta . eps " ) ;

4


6 f i g . plot ( x , y ) ; // OK - style is optional

7 f i g . save ( " da ta . eps " ) ;

8




10 f i g . plot ( x , y , " r " , " Data w / red d o t s " ) ; // *r equals matlab r*11 f i g . save ( " da ta . eps " ) ;

Fig. 12 Fig. 13

Fig. 14

See below for the various colors and linestyles that

can be chosen for plots.

(0.3.11) The plot3 method

This method can visualize curves in 3D space.

Definition:

template

void plot3( const xVector& x,

const yVector& y,

const zVector& z,




Restrictions: Same restrictions as in plot for two vectors, extended to zVector.

Examples:


2 f i g . plot3 ( x , y , z ) ;

3 f i g . save ( " t r a j e c t o r i e s . eps " ) ;

Fig. 15

This is the default color and line style.

As in the case of the plot method different colors

and linestyles can be selected through the optional

string argument style, see 0.3.18.

(0.3.12) The fplot method

This command can be used to plot a function of x passed a expression in a string.

Definition:

void fplot( const std::string& function,


Restrictions: None.

Examples:


2 f i g . f p lo t ( " (3 x ^2 4 . 5 / x ) exp(x / 1 . 3 ) " ) ;3 f i g . f p lo t ( " 5 s i n (5 x ) exp(x ) " , " r " ) . l a b e l ( " 5 s i n ( 5 x ) e^{ x } " ) ;4 f i g . ranges ( 0 .5 , 5 , 5, 5) ; // be sure to set ranges for fplot!5 f i g . save ( " p l o t . eps " ) ;

6


8 f i g . p l o t ( x , y , " b " ) . l a b e l ( " Benchmark " ) ;

9 f i g . f p lo t ( " x ^2 " , " k ; " ) . l a b e l ( "O( x ^ 2 ) " ) ;

10 // here we dont set the ranges as it uses the range given by the

11 // x,y data and we use fplot to draw a reference line O(x2)

12 f i g . save ( " r u n t i m e s . eps " ) ;



Fig. 16 Fig. 17

(0.3.13) The ranges method

The method sets the axis ranges for a 2D plot.

Definition:

void ranges( const double& xMin,

const double& xMax,

const double& yMin,

const double& yMax )

Restrictions: xMin < xMax, yMin < yMax and ranges must be > 0 for axis in logarithmic scale.

Examples:


2 f i g . ranges (1,1 ,1,1) ;3 f i g . p l o t ( x , y , " b " ) ;

4


6 f i g . p l o t ( x , y , " b " ) ;

7 f i g . ranges ( 0 , 2 .3 , 4 , 5) ; // ranges can be called before or after plot

8


10 f i g . ranges(1, 1 , 0 , 5) ;11 f i g . se t log ( true , true ) ; // will run but MathGL will throw a warning

12 f i g . p l o t ( x , y , " b " ) ;



Fig. 18 Fig. 19

Fig. 20

Illegal ranges for logarithmic scale yield a mangled

output.

(0.3.14) The save method

Saves the graphics currently stored in a figure object to file. The default format is EPS.

Definition:

void save( const std::string& file )

Restrictions: Supported file formats: .eps and .png.

Examples:


2 f i g . save ( " p l o t . eps " ) ; // OK

3




5 f i g . save ( " p l o t " ) ; // OK - will be saved as plot.eps

6


8 f i g . save ( " p l o t . png " ) ; // OK - but needs -lpng flag!

(0.3.15) The spy method

This method renders the sparsity pattern of a matrix.

Definition:

template

MglPlot& spy(const Matrix& A, const std::string& style = "b");

template

MglPlot& spy(const Eigen::SparseMatrix& A, const

std::string& style = "b");

Restrictions: None.

Examples:


2 f i g . spy (A) ; // e.g: A = Eigen::MatrixXd, or Eigen::SparseMatrix

Fig. 21 Fig. 22

(0.3.16) The triplot method

This method allows to plot triangulations.

Definition:



template

MglPlot& triplot(const Eigen::Matrix& T,

const xVector& x, const yVector& y, std::string style = "");

template

MglPlot& triplot(const Eigen::Matrix& T,

const xVector& x, const yVector& y, std::string style = "");

Restrictions: The vectors x and ymust have the same dimensions and the matrix Tmust be of dimension

N 3, with N being the number of edges.Examples:


2 f i g . t r i p l o t (T , x , y , " b? " ) ; // ? enumerates all vertices of the mesh

3


5 f i g . t r i p l o t (T , x , y , " bF " ) ; // F will yield a solid background

6 f i g . t r i p l o t (T , x , y , " k " ) ; // draw black mesh on top

Fig. 23 Fig. 24

(0.3.17) The title method

Add a title line to any plot.

Definition:

void title( const std::string&

Date post:	03-Jan-2017
Category:	Documents
Upload:	dangcong
View:	310 times
Download:	2 times

Numerical Methods for Computational Science and Engineering

Documents