Fields Institute Monographs

Fields Institute Monographs

VOLUME 29

For further volumes:http://www.springer.com/series/10502

The Fields Institute is a centre for research in the mathematical sciences, located inToronto, Canada. The Institutes mission is to advance global mathematical activityin the areas of research, education and innovation. The Fields Institute is supportedby the Ontario Ministry of Training, Colleges and Universities, the Natural Sciencesand Engineering Research Council of Canada, and seven Principal SponsoringUniversities in Ontario (Carleton, McMaster, Ottawa, Toronto, Waterloo, Westernand York), as well as by a growing list of Affiliate Universities in Canada, the U.S.and Europe, and several commercial and industrial partners.

The Fields Institute for Research in Mathematical Sciences

Fields Institute Editorial Board:

Carl R. Riehm, Managing Editor

Edward Bierstone, Director of the Institute

Matheus Grasselli, Deputy Director of the Institute

James G. Arthur, University of Toronto

Kenneth R. Davidson, University of Waterloo

Lisa Jeffrey, University of Toronto

Barbara Lee Keyfitz, Ohio State University

Thomas S. Salisbury, York University

Noriko Yui, Queen’s University

Nizar Touzi

Optimal Stochastic Control,Stochastic Target Problems,and Backward SDE

With Chapter 13 by Agnes Tourin

The Fields Institute for Researchin the Mathematical Sciences 123

Nizar TouziDepartement de Mathematiques AppliqueesEcole PolytechniquePalaiseau CedexFrance

ISSN 1069-5273 ISSN 2194-3079 (electronic)ISBN 978-1-4614-4285-1 ISBN 978-1-4614-4286-8 (eBook)DOI 10.1007/978-1-4614-4286-8Springer New York Heidelberg Dordrecht London

Library of Congress Control Number: 2012943958

© Springer Science+Business Media New York 2013This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting, reproduction on microfilms or in any other physical way, and transmission or informationstorage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodologynow known or hereafter developed. Exempted from this legal reservation are brief excerpts in connectionwith reviews or scholarly analysis or material supplied specifically for the purpose of being enteredand executed on a computer system, for exclusive use by the purchaser of the work. Duplication ofthis publication or parts thereof is permitted only under the provisions of the Copyright Law of thePublisher’s location, in its current version, and permission for use must always be obtained from Springer.Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violationsare liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoes not imply, even in the absence of a specific statement, that such names are exempt from the relevantprotective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.

Cover illustration: Drawing of J.C. Fields by Keith Yeomans

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Mathematics Subject Classification (2010): 03C64, 14P15, 26A12, 26A93, 32C05, 32S65, 34C08,34M40, 37S75, 58A17

I should like to expressall my love to my family:

Christine, our sons Ali and Heni, and our daughter Lilia,who accompanied me during this visit to Toronto,

all my thanks to them for their patience while I was preparingthese notes,

and all my apologies for my absence even when I wasphysically present...

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Conditional Expectation and Linear Parabolic PDEs . . . . . . . . . . . . . . . . . . 52.1 Stochastic Differential Equations .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Markovian Solutions of SDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Connection with Linear Partial Differential Equations .. . . . . . . . . . . . 11

2.3.1 Generator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3.2 Cauchy Problem and the Feynman–Kac

Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.3 Representation of the Dirichlet Problem . . . . . . . . . . . . . . . . . . 14

2.4 The Black–Scholes Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.4.1 The Continuous-Time Financial Market. . . . . . . . . . . . . . . . . . . 152.4.2 Portfolio and Wealth Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.4.3 Admissible Portfolios and No-Arbitrage . . . . . . . . . . . . . . . . . . 182.4.4 Super-Hedging and No-Arbitrage Bounds . . . . . . . . . . . . . . . . 182.4.5 The No-Arbitrage Valuation Formula . . . . . . . . . . . . . . . . . . . . . 192.4.6 PDE Characterization of the Black–Scholes Price . . . . . . . . 20

3 Stochastic Control and Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . 213.1 Stochastic Control Problems in Standard Form.. . . . . . . . . . . . . . . . . . . . 213.2 The Dynamic Programming Principle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.1 A Weak Dynamic Programming Principle . . . . . . . . . . . . . . . . 253.2.2 Dynamic Programming Without Measurable Selection . . 27

3.3 The Dynamic Programming Equation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.4 On the Regularity of the Value Function .. . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4.1 Continuity of the Value Function for BoundedControls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4.2 A Deterministic Control Problem with Non-smoothValue Function .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.4.3 A Stochastic Control Problem with Non-smoothValue Function .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

vii

viii Contents

4 Optimal Stopping and Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . 394.1 Optimal Stopping Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.2 The Dynamic Programming Principle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.3 The Dynamic Programming Equation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.4 Regularity of the Value Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.4.1 Finite Horizon Optimal Stopping .. . . . . . . . . . . . . . . . . . . . . . . . . 454.4.2 Infinite Horizon Optimal Stopping . . . . . . . . . . . . . . . . . . . . . . . . 474.4.3 An Optimal Stopping Problem with Nonsmooth

Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Solving Control Problems by Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.1 The Verification Argument for Stochastic Control Problems . . . . . . 535.2 Examples of Control Problems with Explicit Solutions . . . . . . . . . . . . 57

5.2.1 Optimal Portfolio Allocation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.2.2 Law of Iterated Logarithm for Double

Stochastic Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.3 The Verification Argument for Optimal Stopping Problems . . . . . . . 625.4 Examples of Optimal Stopping Problems with Explicit

Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.4.1 Perpetual American Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.4.2 Finite Horizon American Options . . . . . . . . . . . . . . . . . . . . . . . . . 66

6 Introduction to Viscosity Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.1 Intuition Behind Viscosity Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.2 Definition of Viscosity Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686.3 First Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696.4 Comparison Result and Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.4.1 Comparison of Classical Solutions in a BoundedDomain .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.4.2 Semijets Definition of Viscosity Solutions . . . . . . . . . . . . . . . . 746.4.3 The Crandall–Ishii’s Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756.4.4 Comparison of Viscosity Solutions in a Bounded

Domain .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766.5 Comparison in Unbounded Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.6 Useful Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.7 Proof of the Crandall–Ishii’s Lemma. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

7 Dynamic Programming Equation in the Viscosity Sense . . . . . . . . . . . . . . . 897.1 DPE for Stochastic Control Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897.2 DPE for Optimal Stopping Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3 A Comparison Result for Obstacle Problems . . . . . . . . . . . . . . . . . . . . . . . 98

8 Stochastic Target Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.1 Stochastic Target Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

8.1.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.1.2 Geometric Dynamic Programming Principle. . . . . . . . . . . . . . 102

Contents ix

8.1.3 The Dynamic Programming Equation .. . . . . . . . . . . . . . . . . . . . 1048.1.4 Application: Hedging Under Portfolio Constraints . . . . . . . 110

8.2 Stochastic Target Problem with Controlled Probabilityof Success. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1128.2.1 Reduction to a Stochastic Target Problem .. . . . . . . . . . . . . . . . 1138.2.2 The Dynamic Programming Equation .. . . . . . . . . . . . . . . . . . . . 1148.2.3 Application: Quantile Hedging in the Black–Scholes

Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

9 Second Order Stochastic Target Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1239.1 Superhedging Under Gamma Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . 123

9.1.1 Problem Formulation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1249.1.2 Hedging Under Upper Gamma Constraint . . . . . . . . . . . . . . . . 1269.1.3 Including the Lower Bound on the Gamma . . . . . . . . . . . . . . . 132

9.2 Second Order Target Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1349.2.1 Problem Formulation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1349.2.2 The Geometric Dynamic Programming . . . . . . . . . . . . . . . . . . . 1369.2.3 The Dynamic Programming Equation .. . . . . . . . . . . . . . . . . . . . 137

9.3 Superhedging Under Illiquidity Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

10 Backward SDEs and Stochastic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14910.1 Motivation and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

10.1.1 The Stochastic Pontryagin Maximum Principle. . . . . . . . . . . 15010.1.2 BSDEs and Stochastic Target Problems .. . . . . . . . . . . . . . . . . . 15210.1.3 BSDEs and Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

10.2 Wellposedness of BSDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15410.2.1 Martingale Representation for Zero Generator . . . . . . . . . . . . 15410.2.2 BSDEs with Affine Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15510.2.3 The Main Existence and Uniqueness Result . . . . . . . . . . . . . . 156

10.3 Comparison and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15910.4 BSDEs and Stochastic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16010.5 BSDEs and Semilinear PDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16210.6 Appendix: Essential Supremum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

11 Quadratic Backward SDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16511.1 A Priori Estimates and Uniqueness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

11.1.1 A Priori Estimates for Bounded Y . . . . . . . . . . . . . . . . . . . . . . . . . 16611.1.2 Some Properties of BMO Martingales.. . . . . . . . . . . . . . . . . . . . 16711.1.3 Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

11.2 Existence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16911.2.1 Existence for Small Final Condition .. . . . . . . . . . . . . . . . . . . . . . 16911.2.2 Existence for Bounded Final Condition . . . . . . . . . . . . . . . . . . . 172

11.3 Portfolio Optimization Under Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 17511.3.1 Problem Formulation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17511.3.2 BSDE Characterization.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

x Contents

11.4 Interacting Investors with Performance Concern . . . . . . . . . . . . . . . . . . . 18111.4.1 The Nash Equilibrium Problem .. . . . . . . . . . . . . . . . . . . . . . . . . . . 18111.4.2 The Individual Optimization Problem .. . . . . . . . . . . . . . . . . . . . 18211.4.3 The Case of Linear Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18311.4.4 Nash Equilibrium Under Deterministic Coefficients . . . . . 186

12 Probabilistic Numerical Methods for Nonlinear PDEs . . . . . . . . . . . . . . . . . 18912.1 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19012.2 Convergence of the Discrete-Time Approximation .. . . . . . . . . . . . . . . . 19312.3 Consistency, Monotonicity and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19512.4 The Barles–Souganidis Monotone Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 197

13 Introduction to Finite Differences Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20113.1 Overview of the Barles–Souganidis Framework .. . . . . . . . . . . . . . . . . . . 20113.2 First Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

13.2.1 The Heat Equation: The Classic Explicitand Implicit Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

13.2.2 The Black–Scholes–Merton PDE . . . . . . . . . . . . . . . . . . . . . . . . . . 20613.3 A Nonlinear Example: The Passport Option . . . . . . . . . . . . . . . . . . . . . . . . 206

13.3.1 Problem Formulation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20613.3.2 Finite Difference Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 20713.3.3 Howard Algorithm.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

13.4 The Bonnans–Zidani [7] Approximation.. . . . . . . . . . . . . . . . . . . . . . . . . . . 20913.5 Working in a Finite Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21113.6 Variational Inequalities and Splitting Methods . . . . . . . . . . . . . . . . . . . . . 211

13.6.1 The American Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Chapter 1Introduction

These notes have been prepared for the graduate course taught at the FieldsInstitute, Toronto, during the thematic program on quantitative finance which washeld from January to June, 2010.

I would like to thank all the participants to these lectures. It was a pleasure for meto share my experience on this subject with the excellent audience that was offeredby this special research semester. In particular, their remarks and comments helpedto improve parts of this document and to correct some mistakes.

My special thanks go to Bruno Bouchard, Mete Soner, and Agnes Tourin whoaccepted to act as guest lecturers within this course. These notes have also benefittedfrom the discussions with them, and some parts are based on my previous work withBruno and Mete.

These notes have also benefitted from careful reading by Matheus Grasselli,Pierre Henry-Labordere and Tom Salisbury. I greatly appreciate their help and hopethere are not many mistakes left.

I would like to express all my thanks to Matheus Grasselli, Tom Hurd, TomSalisbury, and Sebastian Jaimungal for the warm hospitality at the Fields Instituteand their regular attendance to my lectures.

These lectures present the modern approach to stochastic control problems witha special emphasis on the application in financial mathematics. For pedagogicalreason, we restrict the scope of the course to the control of diffusion processes, thusignoring the presence of jumps.

We first review the main tools from stochastic analysis: Brownian motion andthe corresponding stochastic integration theory. This already introduces to the firstconnection with partial differential equations (PDEs). Indeed, by Ito’s formula,a linear PDE pops up as the infinitesimal counterpart of the tower property.Conversely, given a nicely behaved smooth solution, the so-called Feynman–Kacformula provides a stochastic representation in terms of a conditional expectation.

N. Touzi, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE,Fields Institute Monographs 29, DOI 10.1007/978-1-4614-4286-8 1,© Springer Science+Business Media New York 2013

1

2 1 Introduction

We then introduce the class of standard stochastic control problems whereone wishes to maximize the expected value of some gain functional. The firstmain task is to derive an original weak dynamic programming principle whichavoids the heavy measurable selection arguments in typical proofs of the dynamicprogramming principle when no a priori regularity of the value function is known.The infinitesimal counterpart of the dynamic programming principle is now anonlinear PDE which is called dynamic programming equation or Hamilton–Jacobi–Bellman equation. The hope is that the dynamic programming equationprovides a complete characterization of the problem, once complemented withappropriate boundary conditions. However, this requires strong smoothness con-ditions, which can be seen to be violated in simple examples.

A parallel picture can be drawn for optimal stopping problems and, in fact, forthe more general control and stopping problems. In these notes we do not treatsuch mixed control problems, and we rather analyze separately these two classesof control problems. Here again, we derive the dynamic programming principleand the corresponding dynamic programming equation under strong smoothnessconditions. In the present case, the dynamic programming equation takes the formof the obstacle problem in PDEs.

When the dynamic programming equation happens to have an explicit smoothsolution, the verification argument allows to verify whether this candidate indeedcoincides with the value function of the control problem. The verification argumentprovides as a by-product an access to the optimal control, i.e., the solution of theproblem. But of course, such lucky cases are rare, and one should not count onsolving any stochastic control problem by verification.

In the absence of any general a priori regularity of the value function, the nextdevelopment of the theory is based on viscosity solutions. This beautiful notion wasintroduced by Crandal and Lions and provides a weak notion of solutions to second-order degenerate elliptic PDEs. We review the main tools from viscosity solutionswhich are needed in stochastic control. In particular, we provide a difficulty-incremental presentation of the comparison result (i.e., maximum principle) whichimplies uniqueness.

We next show that the weak dynamic programming equation implies that thevalue function is a viscosity solution of the corresponding dynamic programmingequation in a wide generality. In particular, we do not assume that the controls arebounded. We emphasize that in the present setting, there is no a priori regularityof the value function needed to derive the dynamic programming equation: weonly need it to be locally bounded! Given the general uniqueness results, viscositysolutions provide a powerful tool for the characterization of stochastic control andoptimal stopping problems.

The remaining part of the lectures focus on the more recent literature onstochastic control, namely stochastic target problems. These problems are motivatedby the superhedging problem in financial mathematics. Various extensions havebeen studied in the literature. We focus on a particular setting where the proofsare simplified while highlighting the main ideas.

1 Introduction 3

The use of viscosity solutions is crucial for the treatment of stochastic targetproblems. Indeed, deriving any a priori regularity seems to be a very difficult task.Moreover, by writing formally the corresponding dynamic programming equationand guessing an explicit solution (in some lucky case), there is no known directverification argument as in standard stochastic control problems. Our approach isthen based on a dynamic programming principle suited to this class of problems,and called geometric dynamic programming principle, due to a further extension ofstochastic target problems to front propagation problems in differential geometry.The geometric programming principle allows to obtain a dynamic programmingequation in the sense of viscosity solutions. We provide some examples where theanalysis of the dynamic programming equation leads to a complete solution of theproblem.

We also present an interesting extension to stochastic target problems with con-trolled probability of success. A remarkable trick allows to reduce these problems tostandard stochastic target problems. By using this methodology, we show how onecan solve explicitly the problem of quantile hedging which was previously solved byFollmer and Leukert [21] by duality methods in the standard linear case in financialmathematics.

A further extension of stochastic target problems consists in involving thequadratic variation of the control process in the controlled state dynamics. Theseproblems are motivated by examples from financial mathematics related to marketilliquidity and are called second-order stochastic target problems. We follow thesame line of arguments by formulating a suitable geometric dynamic programmingprinciple and deriving the corresponding dynamic programming equation in thesense of viscosity solutions. The main new difficulty here is to deal with the short-time asymptotics of double stochastic integrals.

The final part of the lectures explores a special type of stochastic targetproblems in the non-Markovian framework. This leads to the theory of backwardstochastic differential equations (BSDE) which was introduced by Pardoux andPeng [32]. Here, in contrast to stochastic target problems, we insist on the existenceof a solution to the stochastic target problem. We provide the main existence,uniqueness, stability, and comparison results. We also establish the connection withstochastic control problems. We finally show the connection with semilinear PDEsin the Markovian case.

The extension of the theory of BSDEs to the case where the generator isquadratic in the control variable is very important in view of the applicationsto portfolio optimization problems. However, the existence and uniqueness cannot be addressed as simply as in the Lipschitz case. The first existence anduniqueness results were established by Kobylanski [26] by adapting to the non-Markovian framework techniques developed in the PDE literature. Instead of thishighly technical argument, we report the beautiful argument recently developed byTevzadze [39] and provide applications in financial mathematics.

The final chapter is dedicated to numerical methods for nonlinear PDEs. Weprovide a complete proof of convergence based on the Barles–Souganidis motone

4 1 Introduction

scheme method. The latter is a beautiful and simple argument which exploitsthe stability of viscosity solutions. Stronger results are provided in the semilinearcase by using techniques from BSDEs.

Chapter 2Conditional Expectation and LinearParabolic PDEs

Throughout this chapter, .�;F ;F; P / is a filtered probability space with filtrationF D fFt ; t � 0g satisfying the usual conditions. Let W D fWt; t � 0g be aBrownian motion valued in R

d , defined on .�;F ;F; P /.Throughout this chapter, a maturity T > 0 will be fixed. By H

2, we denotethe collection of all progressively measurable processes � with appropriate (finite)dimension such that EŒ

R T

0j�t j2dt � < 1.

2.1 Stochastic Differential Equations

In this section, we recall the basic tools from stochastic differential equations

dXt D bt .Xt /dt C �t .Xt/dWt ; t 2 Œ0; T �; (2.1)

where T > 0 is a given maturity date. Here, b and � are F ˝ B.Rn/-progressivelymeasurable functions from Œ0; T � � � � R

n to Rn and MR.n; d/, respectively.

In particular, for every fixed x 2 Rn, the processes fbt .x/; �t .x/; t 2 Œ0; T �g are

F�progressively measurable.

Definition 2.1. A strong solution of (2.1) is an F�progressively measurable pro-cess X such that

R T

0.jbt .Xt/j C j�t .Xt/j2/dt < 1, a.s., and

Xt D X0 CZ t

0

bs.Xs/ds CZ t

0

�s.Xs/dWs; t 2 Œ0; T �:

Let us mention that there is a notion of weak solutions, which relaxes some condi-tions from the above definition in order to allow for more general stochastic differen-tial equations. Weak solutions, as opposed to strong solutions, are defined on someprobabilistic structure (which becomes part of the solution), and not necessarilyon .�;F ;F;P; W /. Thus, for a weak solution we search for a probability structure


5

6 2 Conditional Expectation and Linear Parabolic PDEs

. Q�; QF ; QF; QP; QW / and a process QX such that the requirement of the above definitionholds true. Obviously, any strong solution is a weak solution, but the opposite claimis false.

The main existence and uniqueness result is the following.

Theorem 2.2. Let X0 2 L2 be a r.v. independent of W . Assume that the processes

b:.0/ and �:.0/ are in H2 and that for some K > 0,

jbt .x/ � bt .y/j C j�t .x/ � �t .y/j � Kjx � yj for all t 2 Œ0; T �; x; y 2 Rn:

Then, for all T > 0, there exists a unique strong solution of (2.1) in H2. Moreover,

E

"

supt�T

jXt j2#

� C�1 C EjX0j2

�eC T ; (2.2)

for some constant C D C.T; K/ depending on T and K .

Proof. We first establish the existence and uniqueness result, then we prove theestimate (2.2).

Step 1. For a constant c > 0, to be fixed later, we introduce the norm

k�kH2c

WD E

�Z T

0

e�ct j�t j2dt

�1=2

for every � 2 H2:

Clearly, the norms k:kH2 and k:kH2c

on the Hilbert space H2 are equivalent. Considerthe map U on H

2 by

U.X/t WD X0 CZ t

0

bs.Xs/ds CZ t

0

�s.Xs/dWs; 0 � t � T:

By the Lipschitz property of b and � in the x�variable and the fact that b:.0/�:.0/ 2H

2, it follows that this map is well defined on H2. In order to prove existence and

uniqueness of a solution for (2.1), we shall prove that U.X/ 2 H2 for all X 2 H

2

and that U is a contracting mapping with respect to the norm k:kH2c

for a convenientchoice of the constant c > 0.

1. We first prove that U.X/ 2 H2 for all X 2 H

2. To see this, we decompose:

kU.X/k2H2 � 3T kX0k2

L2 C 3TE

"Z T

0

ˇˇˇˇ

Z t

0

bs.Xs/ds

ˇˇˇˇ

2

dt

#

C3E

"Z T

0

ˇˇˇˇ

Z t

0

�s.Xs/dWs

ˇˇˇˇ

2

dt

#

:

2.1 Stochastic Differential Equations 7

By the Lipschitz continuity of b and � in x, uniformly in t , we have jbt .x/j2 �K.1Cjbt .0/j2Cjxj2/ for some constant K . We then estimate the second term by:

E

"Z T

0

ˇˇˇˇ

Z t

0

bs.Xs/ds

ˇˇˇˇ

2

dt

#

� KTE

�Z T

0

.1 C jbt .0/j2 C jXsj2/ds

�

< 1;

since X 2 H2, and b.:; 0/ 2 L

2.Œ0; T �/.As, for the third term, we use the Doob maximal inequality together with

the fact that j�t .x/j2 � K.1 C j�t .0/j2 C jxj2/, a consequence of the Lipschitzproperty on � :

E

"Z T

0

ˇˇˇˇ

Z t

0

�s.Xs/dWs

ˇˇˇˇ

2

dt

#

� TE

"

maxt�T

ˇˇˇˇ

Z t

0

�s.Xs/dWs

ˇˇˇˇ

2

dt

#

� 4TE

�Z T

0

j�s.Xs/j2ds

�

� 4TKE

�Z T

0

.1 C j�s.0/j2 C jXsj2/ds

�

< 1:

2. To see that U is a contracting mapping for the norm k:kH2c, for some convenient

choice of c > 0, we consider two process X; Y 2 H2 with X0 D Y0, and we

estimate that:

E jU.X/t � U.Y /t j2

� 2E

ˇˇˇˇ

Z t

0

.bs.Xs/ � bs.Ys// ds

ˇˇˇˇ

2

C 2E

ˇˇˇˇ

Z t

0

.�s.Xs/ � �s.Ys// dWs

ˇˇˇˇ

2

D 2E

ˇˇˇˇ

Z t

0

.bs.Xs/ � bs.Ys// ds

ˇˇˇˇ

2

C 2E

Z t

0

j�s.Xs/ � �s.Ys/j2 ds

� 2tE

Z t

0

jbs.Xs/ � bs.Ys/j2 ds C 2E

Z t

0

j�s.Xs/ � �s.Ys/j2 ds

� 2.T C 1/K

Z t

0

E jXs � Ysj2 ds:

Hence, kU.X/ � U.Y /kc � 2K.T C 1/

ckX � Y kc , and therefore U is a

contracting mapping for sufficiently large c.

Step 2. We next prove the estimate (2.2). We shall alleviate the notation writingbs WD bs.Xs/ and �s WD �s.Xs/. We directly estimate


E

�

supu�t

jXuj2�

D E

"

supu�t

ˇˇˇˇX0 C

Z u

0

bsds CZ u

0

�sdWs

ˇˇˇˇ

2#

� 3

EjX0j2 C tE

�Z t

0

jbsj2ds

�

C E

"

supu�t

ˇˇˇˇ

Z u

0

�sdWs

ˇˇˇˇ

2#!

� 3

�

EjX0j2 C tE

�Z t

0

jbsj2ds

�

C 4E

�Z t

0

j�sj2ds

��

where we used the Doob’s maximal inequality. Since b and � are Lipschitzcontinuous in x, uniformly in t and !, this provides:

E

�

supu�t

jXuj2�

� C.K; T /

�

1 C EjX0j2 CZ t

0

E

�

supu�s

jXuj2�

ds

�

;

and we conclude by using the Gronwall lemma. utThe following exercise shows that the Lipschitz-continuity condition on the coef-

ficients b and � can be relaxed. We observe that further relaxation of this assumptionis possible in the one-dimensional case, see, e.g., Karatzas and Shreve [23].

Exercise 2.3. In the context of this section, assume that the coefficients � and � arelocally Lipschitz and linearly growing in x, uniformly in .t; !/. By a localizationargument, prove that strong existence and uniqueness holds for the stochasticdifferential equation (2.1).

In addition to the estimate (2.2) of Theorem 2.2, we have the following flowcontinuity results of the solution of the SDE. In order to emphasize the dependenceon the initial date, we denote by fXt;x

s ; s � tg the solution of the SDE (2.1) withinitial condition X

t;xt D x.

Theorem 2.4. Let the conditions of Theorem 2.2 hold true, and consider some.t; x/ 2 Œ0; T / � R

n with t � t 0 � T .

(i) There is a constant C such that

E

"

supt�s�t 0

ˇˇXt;x

s � Xt;x0

s j2ˇˇ#

� C eC t 0jx � x0j2: (2.3)

(ii) Assume further that B WD supt<t 0�T .t 0�t/�1ER t 0

t

�jbr.0/j2Cj�r .0/j2�dr < 1.Then for all t 0 2 Œt; T �

E

"

supt 0�s�T

ˇˇXt;x

s � Xt 0;xs j2ˇˇ

#

� C eC T .B C jxj2/jt 0 � t j: (2.4)

2.1 Stochastic Differential Equations 9

Proof. (i) To simplify the notations, we set Xs WD Xt;xs and X 0

s WD Xt;x0

s for alls 2 Œt; T �. We also denote ıx WD x � x0, ıX WD X � X 0, ıb WD b.X/ � b.X 0/and ı� WD �.X/ � �.X 0/. We first decompose

jıXsj2 � 3

�

jıxj2 Cˇˇˇ

Z s

t

ıbuduˇˇˇ2 C

ˇˇˇ

Z s

t

ı�udWu

ˇˇˇ2�

� 3

�

jıxj2 C .s � t/

Z s

t

ˇˇıbu

ˇˇ2du C

Z s

t

ı�udWu

ˇˇˇ2�

:

Then, it follows from the Doob maximal inequality and the Lipschitz propertyof the coefficients b and � that

h.t 0/ WD E

"

supt�s�t 0

jıXsj2#

� 3

�

jıxj2 C .s � t/

Z s

t

Eˇˇıbu

ˇˇ2du

C4

Z s

t

Eˇˇı�u

ˇˇ2du

�

� 3

�

jıxj2 C K2.t 0 C 4/

Z s

t

EjıXuj2du

�

� 3

�

jıxj2 C K2.t 0 C 4/

Z s

t

h.u/du

�

:

Then the required estimate follows from the Gronwall inequality.(ii) We next prove (2.4). We again simplify the notation by setting Xs WD Xt;x

s ,s 2 Œt; T �, and X 0

s WD Xt 0;xs , s 2 Œt 0; T �. We also denote ıt WD t 0 � t , ıX WD

X � X 0, ıb WD b.X/ � b.X 0/, and ı� WD �.X/ � �.X 0/. Then following thesame arguments as in the previous step, we obtain for all u 2 Œt 0; T �:

h.u/ WD E

"

supt 0�s�u

jıXsj2#

� 3

�

EjXt 0 � xj2 C K2.T C 4/

Z u

t 0

EjıXr j2dr

�

� 3

�

EjXt 0 � xj2 C K2.T C 4/

Z u

t 0

h.r/dr

�

(2.5)

Observe that

EjXt 0 � xj2 � 2

E

ˇˇˇ

Z t 0

t

br .Xr/drˇˇˇ2 C E

ˇˇˇ

Z t 0

t

�r .Xr/drˇˇˇ2

!

� 2

T

Z t 0

t

Ejbr.Xr/j2dr CZ t 0

t

Ej�r.Xr/j2dr

!


� 6.T C 1/

Z t 0

t

�K2

EjXr � xj2 C jxj2 C Ejbr.0/j2�dr

� 6.T C 1/�.t 0 � t/.jxj2 C B/ C K2

Z t 0

t

EjXr � xj2dr:

By the Gronwall inequality, this shows that

EjXt 0 � xj2 � C.jxj2 C B/jt 0 � t jeC.t 0�t /:

Plugging this estimate in (2.5), we see that

h.u/ � 3

�

C.jxj2 C B/jt 0 � t jeC .t 0 � t/ C K2.T C 4/

Z u

t 0

h.r/dr

�

; (2.6)

and the required estimate follows from the Gronwall inequality. ut

2.2 Markovian Solutions of SDEs

In this section, we restrict the coefficients b and � to be deterministic functions of.t; x/. In this context, we write

bt .x/ D b.t; x/; �t .x/ D �.t; x/ for t 2 Œ0; T �; x 2 Rn;

where b and � are continuous functions, Lipschitz in x uniformly in t . Let Xt;x:

denote the solution of the stochastic differential equation

Xt;xs D x C

Z s

t

b�u; Xt;x

u

�du C

Z s

t

��u; Xt;x

u

�dWu s � t:

The two following properties are obvious:

• Clearly, Xt;xs D F .t; x; s; .W: � Wt/t�u�s/ for some deterministic function F .

• For t � u � s: Xt;xs D Xu;X

t;xu

s . This follows from the pathwise uniqueness, andholds also when u is a stopping time.

With these observations, we have the following Markovian property for the solutionsof stochastic differential equations.

Proposition 2.5 (Strong Markovian property). For all stopping time � withvalues in [0, s]

E Œ˚ .Xu; � � u � s/ jF� � D E Œ˚ .Xu; � � u � s/ jX��

for all bounded function ˚ W C.Œ�; s�/ �! R.

2.3 Connection with Linear Partial Differential Equations 11

2.3 Connection with Linear Partial Differential Equations

2.3.1 Generator

Let fXt;xs ; s � tg be the unique strong solution of

Xt;xs D x C

Z s

t

b.u; Xt;xu /du C

Z s

t

�.u; Xt;xu /dWu; s � t;

where � and � satisfy the required condition for existence and uniqueness of astrong solution.

For a function f W Rn �! R, we define the function Af by

Af .t; x/ D limh!0

EŒf .Xt;xtCh/� � f .x/

hif the limit exists.

Clearly, Af is well defined for all bounded C 2� function with bounded derivativesand

Af .t; x/ D b.t; x/ � Df .x/ C 1

2Tr��T.t; x/D2f .x/

�; (2.7)

where Df and D2f denote the gradient and Hessian of f , respectively. (Exercise !).The linear differential operator A is called the generator of X . It turns out that theprocess X can be completely characterized by its generator or, more precisely, bythe generator and the corresponding domain of definition.

As the following result shows, the generator provides an intimate connectionbetween conditional expectations and linear partial differential equations.

Proposition 2.6. Assume that the function .t; x/ 7�! v.t; x/ WD Eg.Xt;x

T /�

isC 1;2 .Œ0; T / � R

n/. Then v solves the partial differential equation:

@v

@tC Av D 0 and v.T; :/ D g:

Proof. Given .t; x/ with 2ı WD T � t > 0, let �1 WD .T � ı/ ^ inffu > t WjXt;x

u �xj � 1g. For an arbitrary s > t , it follows from the law of iterated expectationtogether with the Markovian property of the process X that

v.t; x/ D Ev�s ^ �1; Xt;x

s^�1

��:

Since v 2 C 1;2.Œ0; T /;Rn/, we may apply Ito’s formula, and we obtain by takingexpectations:


0 D E

�Z s^�1

t

�@v

@tC Av

�

.u; Xt;xu /du

�

CE

�Z s^�1

t

@v

@x.u; Xt;x

s / � �.u; Xt;xu /dWu

�

D E

�Z s^�1

t

�@v

@tC Av

�

.u; Xt;xu /du

�

;

where the last equality follows from the boundedness of .u; Xt;xu / on Œt; s ^ �1�

together with the continuity of � . We now send s & t , and the required resultfollows from the dominated convergence theorem. ut

2.3.2 Cauchy Problem and the Feynman–Kac Representation

In this section, we consider the following linear partial differential equation,

@v

@tC Av � k.t; x/v C f .t; x/ D 0; .t; x/ 2 Œ0; T / � R

d

v.T; :/ D g

; (2.8)

where A is the generator (2.7), g is a given function from Rd to R, k and f are

functions from Œ0; T � � Rd to R, b and � are functions from Œ0; T � � R

d to Rd and

MR.d; d/, respectively. This is the so-called Cauchy problem.For example, when k D f � 0, b � 0, and � is the identity matrix, the above

partial differential equation reduces to the heat equation.Our objective is to provide a representation of this purely deterministic problem

by means of stochastic differential equations. We then assume that b and � satisfythe conditions of Theorem 2.2, namely that

b; � Lipschitz in x uniformly in t ;

Z T

0

�jb.t; 0/j2 C j�.t; 0/j2� dt < 1: (2.9)

Theorem 2.7. Let the coefficients b� be continuous and satisfy (2.9). Assumefurther that the function k is uniformly bounded from below and f has quadraticgrowth in x uniformly in t . Let v be a C 1;2

�Œ0; T /;Rd

�solution of (2.8) with

quadratic growth in x uniformly in t . Then

v.t; x/ D E

�Z T

t

ˇt;xs f .s; Xt;x

s /ds C ˇt;xT g

�X

t;xT

��

; t � T; x 2 Rd ;

where fXt;xs ; s � tg is the solution of the SDE (2.1) with initial data X

t;xt D x, and

ˇt;xs WD e� R s

t k.u;Xt;xu /du for t � s � T .

2.3 Connection with Linear Partial Differential Equations 13

Proof. For n � 1, we denote Tn WD T � 1=n. For t < T , we have Tn > t for largen, and we introduce the sequence of stopping times

�n WD Tn ^ inf˚s > t W ˇ

ˇXt;xs � x

ˇˇ � n

�:

Then �n �! T P�a.s. Since v is smooth, it follows from Ito’s formula that fort � s < T

d�ˇt;x

s v�s; Xt;x

s

�� D ˇt;xs

�

�kv C @v

@tC Av

��s; Xt;x

s

�ds

Cˇt;xs

@v

@x

�s; Xt;x

s

� � ��s; Xt;x

s

�dWs

D ˇt;xs

�

�f .s; Xt;xs /ds C @v

@x

�s; Xt;x

s

� � ��s; Xt;x

s

�dWs

�

;

by the PDE satisfied by v in (2.8). Then:

Eˇt;x

�nv��n; Xt;x

�n

�� v.t; x/

D E

�Z �n

t

ˇt;xs

�

�f .s; Xs/ds C @v

@x

�s; Xt;x

s

� � ��s; Xt;x

s

�dWs

��

:

Now observe that the integrands in the stochastic integral are bounded by definitionof the stopping time �n, the smoothness of v, and the continuity of � and theboundedness of ˇ. Then the stochastic integral has zero mean, and we deduce that

v.t; x/ D E

�Z �n

t

ˇt;xs f

�s; Xt;x

s

�ds C ˇt;x

�nv��n; Xt;x

�n

��

: (2.10)

Since �n �! T and the Brownian motion has continuous sample paths P�a.s., itfollows from the continuity of v that P�a.s.

Z �n

t

ˇt;xs f

�s; Xt;x

s

�ds C ˇt;x

�nv��n; Xt;x

�n

�

n!1�!Z T

t

ˇt;xs f

�s; Xt;x

s

�ds C ˇ

t;xT v

�T; X

t;xT

�

DZ T

t

ˇt;xs f

�s; Xt;x

s

�ds C ˇ

t;xT g

�X

t;xT

�(2.11)

by the terminal condition satisfied by v in (2.8). Moreover, since k is bounded frombelow and the functions f and v have quadratic growth in x uniformly in t , we have

ˇˇˇˇ

Z �n

t

ˇt;xs f

�s; Xt;x

s

�ds C ˇt;x

�nv��n; Xt;x

�n

�ˇˇˇˇ � C

�

1 C maxt�T

jXt j2�

:


By the estimate stated in the existence and uniqueness Theorem 2.2, the latter boundis integrable, and we deduce from the dominated convergence theorem that theconvergence in (2.11) holds in L

1.P/, proving the required result by taking limitsin (2.10). ut

The above Feynman–Kac representation formula has an important numericalimplication. Indeed it opens the door to the use of Monte Carlo methods in orderto obtain a numerical approximation of the solution of the partial differentialequation (2.8). For the sake of simplicity, we provide the main idea in the casef D k D 0. Let

�X.1/; : : : ; X.k/

�be an iid sample drawn in the distribution of X

t;xT ,

and compute the mean:

Ovk.t; x/ WD 1

k

kX

iD1

g�X.i/

�:

By the law of large numbers, it follows that Ovk.t; x/ �! v.t; x/ P�a.s. Moreover,the error estimate is provided by the central limit theorem:

pk .Ovk.t; x/ � v.t; x//

k!1�! N �0;Var

g�Xt;x

T

��in distribution

and is remarkably independent of the dimension d of the variable X !

2.3.3 Representation of the Dirichlet Problem

Let D be an open subset of Rd . The Dirichlet problem is to find a function u solving:

Au � ku C f D 0 on D and u D g on @D; (2.12)

where @D denotes the boundary of D and A is the generator of the process X0;x

defined as the unique strong solution of the stochastic differential equation

X0;xt D x C

Z t

0

�.X0;xs /ds C

Z t

0

�.X0;xs /dWs; t � 0:

Similarly to the representation result of the Cauchy problem obtained in Theo-rem 2.7, we have the following representation result for the Dirichlet problem.

Theorem 2.8. Let u be a C 2�solution of the Dirichlet problem (2.12). Assume thatk is nonnegative, and

EŒ�xD� < 1; x 2 R

d ; where �xD WD inf

nt � 0 W X

0;xt 62 D

o:

2.4 The Black–Scholes Model 15

Then, we have the representation

u.x/ D E

"

g�X

0;x�xD

�e� R �x

D0 k.Xs/ds C

Z �xD

0

f�X

0;xt

�e� R t

0 k.Xs/dsdt

#

:

Exercise 2.9. Provide a proof of Theorem 2.8 by imitating the arguments in theproof of Theorem 2.7.

2.4 The Black–Scholes Model

In this section, we provide a self-contained review of the standard theory of pricingand hedging derivative securities in complete markets. In the context of a Markovianmodel for the underlying assets, the solution is expressed in terms of a parabolicPDE or, equivalently, a conditional expectation operator.

2.4.1 The Continuous-Time Financial Market

Let T be a finite horizon and .�;F ;P/ be a complete probability space supporting aBrownian motion W D f.W 1

t ; : : : ; W dt /, 0 � t � T g with values in R

d . We denoteby F D F

W D fFt ; 0 � t � T g the canonical augmented filtration of W , i.e., thecanonical filtration augmented by zero measure sets of FT .

We consider a financial market consisting of d C 1 assets:

(i) The first asset S0 is locally riskless, and is defined by

S0t D exp

�Z t

0

rudu

�

; 0 � t � T;

where frt ; t 2 Œ0; T �g is a non-negative adapted processes withR T

0 rtdt < 1a.s. and represents the instantaneous interest rate.

(ii) The d remaining assets Si , i D 1; : : : ; d , are risky assets with price processesdefined by the dynamics

dSit

S it

D �it dt C

dX

j D1

�i;jt dW

jt ; t 2 Œ0; T �;

for 1 � i � d , where �; � are F�adapted processes withR T

0 j�it jdt C

R T

0j�i;j

t j2dt < 1 for all i; j D 1; : : : ; d . It is convenient to use the matrixnotations to represent the dynamics of the price vector S D .S1; : : : ; Sd /:


dSt D St ? .�t dt C �t dWt/ ; t 2 Œ0; T �;

where, for two vectors x; y 2 Rd , we denote x ? y the vector of R

d withcomponents .x ? y/i D xi yi , i D 1; : : : ; d , and �; � are the R

d �vector withcomponents �i ’s, and the MR.d; d/�matrix with entries �i;j .

We assume that the MR.d; d/�matrix �t is invertible for every t 2 Œ0; T � a.s.,and we introduce the process

�t WD ��1t .�t � rt1/ ; 0 � t � T;

called the risk premium process. Here 1 is the vector of ones in Rd . We shall

frequently make use of the discounted processes

QSt WD St

S0t

D St exp

�

�Z t

0

rudu

�

Using the above matrix notations, the dynamics of the process QS is given by

d QSt D QSt ?�.�t � rt 1/dt C �t dWt

� D QSt ? �t .�t dt C dWt/ :

2.4.2 Portfolio and Wealth Process

A portfolio strategy is an F�adapted process D ft ; 0 � t � T g with values inR

d . For 1 � i � n and 0 � t � T , it is the amount (in Euros) invested in the risky

asset Si .We next recall the self-financing condition in the present framework. Let X

t

denote the portfolio value, or wealth, process at time t induced by the portfoliostrategy . Then, the amount invested in the non-risky asset is X

t � PniD1 i

t DX

t � t � 1, and the dynamics of the wealth process is given by

dXt D

nX

iD1

it

S it

dSit C X

t � t � 1

S0t

dS0t :

Let QX be the discounted wealth process

QXt WD X

t exp

�

�Z t

0

rudu

�

; 0 � t � T:

Then, by an immediate application of Ito’s formula, we see that

d QXt D Qt � �t .�t dt C dWt/ ; 0 � t � T; (2.13)


where Qt WD e� R t0 rudut . We still need to place further technical conditions on , at

least in order for the above wealth process to be well defined as a stochastic integral.Before this, let us observe that, assuming that the risk premium process satisfies

the Novikov condition:

E

he

12

R T0 j�t j2dt

i< 1;

it follows from the Girsanov theorem that the process

Bt WD Wt CZ t

0

�udu; 0 � t � T; (2.14)

is a Brownian motion under the equivalent probability measure

Q WD ZT � P on FT where ZT WD exp

�

�Z T

0

�u � dWu � 1

2

Z T

0

j�uj2du

�

:

In terms of the Q Brownian motion B , the discounted price process satisfies

d QSt D QSt ? �t dBt ; t 2 Œ0; T �;

and the discounted wealth process induced by an initial capital X0 and a portfoliostrategy can be written in

QXt D QX0 C

Z t

0

Qu � �udBu; for 0 � t � T: (2.15)

Definition 2.10. An admissible portfolio process D ft ; t 2 Œ0; T �g is anF�progressively measurable process such that

R T

0j�T

t t j2dt < 1, a.s., and the cor-responding discounted wealth process is bounded from below by a Q�martingale

QXt � M

t ; 0 � t � T; for some Q�martingale M :

The collection of all admissible portfolio processes will be denoted by A.

The lower bound M , which may depend on the portfolio , has the interpreta-tion of a finite credit line imposed on the investor. This natural generalization of themore usual constant credit line corresponds to the situation where the total creditavailable to an investor is indexed by some financial holding, such as the physicalassets of the company or the personal home of the investor, used as collateral.From the mathematical viewpoint, this condition is needed in order to exclude anyarbitrage opportunity and will be justified in the subsequent subsection.


2.4.3 Admissible Portfolios and No-Arbitrage

We first define precisely the notion of no-arbitrage.

Definition 2.11. We say that the financial market contains no-arbitrage opportuni-ties if for any admissible portfolio process 2 A,

X0 D 0 and XT � 0 P � a.s. implies X

T D 0 P � a.s.

The purpose of this section is to show that the financial market describedabove contains no-arbitrage opportunities. Our first observation is that, by the verydefinition of the probability measure Q, the discounted price process QS satisfies:

the process˚ QSt ; 0 � t � T

�is a Q � local martingale. (2.16)

For this reason, Q is called a risk neutral measure, or an equivalent local martingalemeasure, for the price process S .

We also observe that the discounted wealth process satisfies:

QX is a Q�local martingale for every 2 A; (2.17)

as a stochastic integral with respect to the Q�Brownian motion B .

Theorem 2.12. The continuous-time financial market described above contains no-arbitrage opportunities, i.e., for every 2 A

X0 D 0 and XT � 0 P � a.s. H) X

T D 0 P � a.s.

Proof. For 2 A, the discounted wealth process QX is a Q�local martingalebounded from below by a Q�martingale. Then QX is a Q�super-martingale(Exercise!). In particular, EQ

QXT

� � QX0 D X0. Recall that Q is equivalent to P

and S0 is strictly positive. Then, this inequality shows that, whenever X0 D 0 and

XT � 0 P�a.s. (or equivalently Q�a.s.), we have QX

T D 0 Q�a.s. and thereforeX

T D 0 P�a.s. ut

2.4.4 Super-Hedging and No-Arbitrage Bounds

Let G be an FT �measurable random variable representing the payoff of a derivativesecurity with given maturity T > 0. The super hedging problem consists in findingthe minimal initial cost so as to be able to face the payment G without risk at thematurity T of the contract:

V.G/ WD inf fX0 2 R W XT � G P � a.s. for some 2 Ag :


Remark 2.13. Notice that V.G/ depends on the reference measure P only by meansof the corresponding null sets. Therefore, the super-hedging problem is not changedif P is replaced by any equivalent probability measure.

We now show that, under the no-arbitrage condition, the super hedging problemprovides no-arbitrage bounds on the market price of the derivative security.

Assume that the buyer of the contingent claim G has the same access to thefinancial market than the seller. Then V.G/ is the maximal amount that the buyerof the contingent claim contract is willing to pay. Indeed, if the seller requires apremium of V.G/ C2", for some " > 0, then the buyer would not accept to pay thisamount as he can obtain at least G by trading on the financial market with initialcapital V.G/ C ".

Now, since selling of the contingent claim G is the same as buying the contingentclaim �G, we deduce from the previous argument that

� V.�G/ � market price of G � V.G/: (2.18)

2.4.5 The No-Arbitrage Valuation Formula

We denote by p.G/ the market price of a derivative security G.

Theorem 2.14. Let G be an FT �measurable random variable representing thepayoff of a derivative security at the maturity T > 0, and recall the notationQG WD G exp

�� R T

0rt dt

. Assume that EQŒj QGj� < 1. Then, in the context of our

(complete) financial market:

p.G/ D V.G/ D EQŒ QG�:

Moreover, there exists a portfolio � 2 A such that X�

0 D p.G/ and X�

T D G,a.s., i.e., � is a perfect replication strategy.

Proof. 1. We first prove that V.G/ � EQŒ QG�. Let X0 and 2 A be such that

XT � G, a.s. or, equivalently, QX

T � QG a.s. Notice that QX is a Q�super-martingale, as a Q�local martingale bounded from below by a Q�martingale.Then X0 D QX0 � E

QŒ QXT � � E

QŒ QG�.2. We next prove that V.G/ � E

QŒ QG�. Define the Q�martingale Yt WD EQŒ QGjFt �

and observe that FW D FB . Then, it follows from the martingale representation

theorem that Yt D Y0 C R T

0 �t � dBt for some F�adapted process � withR T

0j�t j2dt < 1 a.s. Setting Q� WD .�T/�1�, we see that

� 2 A and Y0 CZ T

0

Q� � �t dBt D QG P � a.s.

which implies that Y0 � V.G/ and � is a perfect hedging strategy for G, startingfrom the initial capital Y0.


3. From the previous steps, we have V.G/ D EQŒ QG�. Applying this result to �G,

we see that V.�G/ D �V.G/, so that the no-arbitrage bounds (2.18) imply thatthe no-arbitrage market price of G is given by V.G/. ut

2.4.6 PDE Characterization of the Black–Scholes Price

In this subsection, we specialize further the model to the case where the riskysecurities price processes are Markovian diffusions defined by the stochasticdifferential equations:

dSt D St ?�r.t; St /dt C �.t; St /dBt

�:

Here .t; s/ 7�! s ? r.t; s/ and .t; s/ 7�! s ? �.t; s/ are functions from RC �Œ0; 1/d to R

d and Sd , successively satisfying the required continuity and Lipschitzconditions. We also consider a Vanilla derivative security defined by the payoff

G D g.ST /;

where g W Œ0; 1/d ! R is a measurable function with linear growth. By animmediate extension of the results from the previous subsection, the no-arbitrageprice at time t of this derivative security is given by

V.t; St / D EQ

he� R T

t r.u;Su/dug.ST /jFt

iD E

Q

he� R T

t r.u;Su/dug.ST /jSt

i;

where the last equality follows from the Markovian property of the process S . Sinceg has linear growth, it follows that V has linear growth in s uniformly in t . Since V

is defined by a conditional expectation, it is expected to satisfy the linear PDE:

� @t V � rs ? DV � 1

2Tr.s ? �/2D2V

�C rV D 0: (2.19)

More precisely, if V 2 C 1;2.RC;Rd /, then V is a classical solution of (2.19) andsatisfies the final condition V.T; :/ D g. Conversely, if the PDE (2.19) combinedwith the final condition v.T; :/ D g has a classical solution v with linear growth,then v coincides with the derivative security price V as stated by Theorem 2.7.

Chapter 3Stochastic Control and Dynamic Programming

In this chapter, we assume that the filtration F is the P�augmentation of thecanonical filtration of the Brownian motion W . This restriction is only neededin order to simplify the presentation of the proof of the dynamic programmingprinciple. We will also denote by

S WD Œ0; T / � Rd where T 2 Œ0; 1�:

The set S is called the parabolic interior of the state space. We will denote by NS WDcl.S/ its closure, i.e., NS D Œ0; T � � R

d for finite T and NS D S for T D 1.

3.1 Stochastic Control Problems in Standard Form

Control Processes. Given a subset U of Rk , we denote by U the set of all

progressively measurable processes � D f�t ; t < T g valued in U . The elementsof U are called control processes.

Controlled process. Let

b W .t; x; u/ 2 S � U �! b.t; x; u/ 2 Rd

and

� W .t; x; u/ 2 S � U �! �.t; x; u/ 2 MR.n; d/

be two continuous functions satisfying the conditions

jb.t; x; u/ � b.t; y; u/j C j�.t; x; u/ � �.t; y; u/j � K jx � yj; (3.1)

jb.t; x; u/j C j�.t; x; u/j � K .1 C jxj C juj/ (3.2)


21

22 3 Stochastic Control and Dynamic Programming

for some constant K independent of .t; x; y; u/. For each control process � 2 U , weconsider the controlled stochastic differential equation:

dXt D b.t; Xt ; �t /dt C �.t; Xt ; �t /dWt : (3.3)

If the above equation has a unique solution X , for a given initial data, then theprocess X is called the controlled process, as its dynamics is driven by the action ofthe control process �.

We shall be working with the following subclass of control processes:

U0 WD U \ H2; (3.4)

where H2 is the collection of all progressively measurable processes with finite

L2.� � Œ0; T //�norm. Then, for every finite maturity T 0 � T , it follows from

the above uniform Lipschitz condition on the coefficients b and � that

E

"Z T 0

0

�jbj C j� j2� .s; x; �s/ds

#< 1 for all � 2 U0; x 2 R

d ;

which guarantees the existence of a controlled process on the time interval Œ0; T 0�for each given initial condition and control. The following result is an immediateconsequence of Theorem 2.2.

Theorem 3.1. Let � 2 U0 be a control process and � 2 L2.P/ be an

F0�measurable random variable. Then, there exists a unique F�adapted processX� satisfying (3.3) together with the initial condition X�

0 D �. Moreover, for everyT > 0, there is a constant C > 0 such that

E

"sup

0�s�t

jX�s j2#

< C.1 C EŒj�j2�/eC t for all t 2 Œ0; T /: (3.5)

Gain Functional. Let

f; k W Œ0; T / � Rd � U �! R and g W R

d �! R

be given functions. We assume that fandk are continuous and kk�k1 < 1 (i.e.,max.�k; 0/ is uniformly bounded). Moreover, we assume that f and g satisfy thequadratic growth condition:

jf .t; x; u/j C jg.x/j � K.1 C juj C jxj2/;

for some constant K independent of .t; x; u/. We define the gain function J onŒ0; T � � R

d � U by

3.1 Stochastic Control Problems in Standard Form 23

J.t; x; �/ WD E

�Z T

t

ˇ�.t; s/f .s; Xt;x;�s ; �s/ds C ˇ�.t; T /g.X

t;x;�T /1T <1

�;

when this expression is meaningful, where

ˇ�.t; s/ WD e� R st k.r;X

t;x;�r ;�r /dr ;

and fXt;x;�s ; s � tg is the solution of (3.3) with control process � and initial condition

Xt;x;�t D x.

Admissible Control Processes. In the finite horizon case T < 1, the quadraticgrowth condition on f and g together with the bound on k� ensure, that J.t; x; �/

is well defined for all control process � 2 U0. We then define the set of admissiblecontrols in this case by U0.

More attention is needed for the infinite horizon case. In particular, the discountterm k needs to play a role to ensure the finiteness of the integral. In this setting thelargest set of admissible control processes is given by

U0 WD�

� 2 U W E

�Z 1

0

ˇ�.t; s/�1 C jXt;x;�

s j2 C j�s/j�

ds

�

< 1 for all x

�when T D 1:

The Stochastic Control Problem. Consider the optimization problem

V.t; x/ WD sup�2U0

J.t; x; �/ for .t; x/ 2 S:

Our main concern is to describe the local behavior of the value function V

by means of the so-called dynamic programming equation, or Hamilton-Jacobi-Bellman equation. We continue with some remarks.

Remark 3.2. (i) If V.t; x/ D J.t; x; O�t;x/, we call O�t;x an optimal control for theproblem V.t; x/.

(ii) The following are some interesting subsets of controls :

– A process � 2 U0 which is adapted to the natural filtration FX of the

associated state process is called feedback control.– A process � 2 U0 which can be written in the form �s D Qu.s; Xs/ for some

measurable map Qu from Œ0; T � � Rd into U , is called Markovian control;

notice that any Markovian control is a feedback control.– The deterministic processes of U0 are called open loop controls.

(iii) Suppose that T < 1, and let .Y; Z/ be the controlled processes defined by

dYs D Zsf .s; Xs; �s/ds and dZs D �Zsk.s; Xs; �s/ds;


and define the augmented state process NX WD .X; Y; Z/. Then, the above valuefunction V can be written in the form:

V.t; x/ D NV .t; x; 0; 1/ ;

where Nx D .x; y; z/ is some initial data for the augmented state process NX ,

NV .t; Nx/ WD Et; Nx� Ng. NXT /

and Ng.x; y; z/ WD y C g.x/z :

Hence, the stochastic control problem V can be reduced without loss ofgenerality to the case where f D k � 0. We shall appeal to this reduced formwhenever convenient for the exposition.

(iv) Consider the case T < 1. In view of the previous remark, we may assumewithout loss of generality that f D k D 0. Consider the value function

QV .t; x/ WD sup�2Ut

E�g.X

t;x;�T /

; (3.6)

differing from V by the restriction of the control processes to

Ut WD f� 2 U0 W � independent of Ftg : (3.7)

Since Ut � U0, it is obvious that QV � V . We claim that

QV D V; (3.8)

so that both problems are indeed equivalent. To see this, fix .t; x/ 2 S and � 2U0. Then, � can be written as a measurable function of the canonical process�..!s/0�s�t ; .!s � !t /t�s�T /, where, for fixed .!s/0�s�t , the map �.!s/0�s�t W.!s � !t /t�s�T 7! �..!s/0�s�t ; .!s � !t /t�s�T / can be viewed as a controlindependent on Ft . Using the independence of the increments of the Brownianmotion, together with Fubini’s lemma, it thus follows that

J.t; xI �/ DZ

E

hg.X

t;x;�.!s/0�s�t

T /i

dP..!s/0�s�t /

�Z

QV .t; x/dP..!s/0�s�t / D QV .t; x/:

By arbitrariness of � 2 U0, this implies that QV .t; x/ � V.t; x/.(v) The extension of the previous remark to the infinite horizon case is also

immediate.

3.2 The Dynamic Programming Principle 25

3.2 The Dynamic Programming Principle

3.2.1 A Weak Dynamic Programming Principle

The dynamic programming principle is the main tool in the theory of stochasticcontrol. In these notes, we shall prove rigorously a weak version of the dynamicprogramming which will be sufficient for the derivation of the dynamic program-ming equation. We denote:

V�.t; x/ WD lim inf.t 0;x0/!.t;x/

V .t 0; x0/ and V �.t; x/ WD lim sup.t 0;x0/!.t;x/

V .t 0; x0/;

for all .t; x/ 2 NS. The functions V� and V � are called the lower and uppersemicontinuous envelope, respectively. Clearly they are finite whenever the functionV is locally bounded. We also recall the subset of controls Ut introduced in (3.7)above.

Theorem 3.3. Assume that V is locally bounded. Let .t; x/ 2 S be fixed. Letf��; � 2 Utg be a family of finite stopping times independent of Ft with valuesin Œt; T �. Then:

V.t; x/ � sup�2Ut

E

"Z ��

t

ˇ�.t; s/f .s; Xt;x;�s ; �s/ds C ˇ�.t; ��/V �.��; X

t;x;�� /

#:

Assume further that g is lower semicontinuous and X�t;x1Œt;�� is L1�bounded for

all � 2 Ut . Then


E

"Z ��

t

ˇ�.t; s/f .s; Xt;x;�s ; �s/ds C ˇ�.t; ��/V�.��; X

t;x;�� /

#:

We shall provide an intuitive justification of this result after the followingcomments. A rigorous proof is reported in Sect. 3.2.2.

Remark. The possible dependence of � on the central � allows for more flexiblechoices of the stopping times and will be essential for our localization arguments inthe rest of these lectures. In particular, the boundedness condition of X�

t;x˘Œt;� v� willbe automatically satisfied by localization.

Remark. (i) If V is continuous, then V D V� D V �, and the above weak dynamicprogramming principle reduces to the classical dynamic programming principle:

V.t; x/ D sup�2U

Et;x

"Z �

t

ˇ.t; s/f .s; Xs; �s/ds C ˇ.t; �/V .�; X� /

#: (3.9)


(ii) In the discrete-time framework, the dynamic programming principle (3.9) canbe stated as follows :

V.t; x/ D supu2U

Et;x

�f .t; Xt ; u/ C e�k.tC1;XtC1;u/V .t C 1; XtC1/

:

Observe that the supremum is now taken over the subset U of the finite dimen-sional space Rk . Hence, the dynamic programming principle allows to reducethe initial maximization problem, over the subset U of the infinite dimensionalset of Rk�valued processes, into a finite dimensional maximization problem.However, we are still facing an infinite dimensional problem since the dynamicprogramming principle relates the value function at time t to the value functionat time t C 1.

(iii) In the context of the above discrete-time framework with finite horizonT < 1, notice that the dynamic programming principle suggests the followingbackward algorithm to compute V as well as the associated optimal strategy(when it exists). Since V.T; �/ D g is known, the above dynamic programmingprinciple can be applied recursively in order to deduce the value functionV.t; x/ for every t .

(iv) In the continuous time setting, there is no obvious counterpart to the abovebackward algorithm. But, as the stopping time � approaches t , the abovedynamic programming principle implies a special local behavior for the valuefunction V . When V is known to be smooth, this will be obtained by means ofIto’s formula.

(v) It is usually very difficult to determine a priori the regularity of V . Thesituation is even worse since there are many counter-examples showing thatthe value function V can not be expected to be smooth in general; see Sect. 3.4.This problem is solved by appealing to the notion of viscosity solutions, whichprovides a weak local characterization of the value function V , and leadsto the so-called dynamic programming equation or Hamilton-Jacobi-Bellmanequation.

(vi) Once the local behavior of the value function is characterized, we are faced tothe important uniqueness issue, which implies that V is completely character-ized by its local behavior together with some convenient boundary condition.

Intuitive Justification of (3.9). Let us assume that V is continuous. In particular,V is measurable and V D V� D V �. Let QV .t; x/ denote the right-hand side of (3.9).

By the tower property of the conditional expectation operator, it is easilychecked that

J.t; x; �/ D Et;x

"Z �

t

ˇ.t; s/f .s; Xs; �s/ds C ˇ.t; �/J.�; X� ; �/

#:

Since J.�; X� ; �/ � V.�; X�/, this proves that V � QV . To prove the reverseinequality, let � 2 U and " > 0 be fixed, and consider an "�optimal control �"

for the problem V.�; X� /, i.e.,


J.�; X� ; �"/ � V.�; X�/ � ":

Clearly, one can choose �" D � on the stochastic interval Œt; ��. Then

V.t; x/ � J.t; x; �"/ D Et;x

"Z �

t

ˇ.t; s/f .s; Xs; �s/ds C ˇ.t; �/J.�; X� ; �"/

#

� Et;x

"Z �

t

ˇ.t; s/f .s; Xs; �s/ds C ˇ.t; �/V .�; X�/

#� " Et;x Œˇ.t; �/� :

This provides the required inequality by the arbitrariness of � 2 U and " > 0. utExercise. Where is the gap in the above sketch of the proof ?

Answer: The existence of the above control process �" is not obvious because ofcrucial measurability problems!

3.2.2 Dynamic Programming Without Measurable Selection

In this section, we provide a rigorous proof of Theorem 3.3. Notice that, we have noinformation on whether V is measurable or not. Because of this, the right-hand sideof the classical dynamic programming principle (3.9) is not even known to be welldefined.

The formulation of Theorem 3.3 avoids this measurability problem since V� andV � are lower and upper semicontinuous, respectively, and therefore measurable. Inaddition, it allows to avoid the typically heavy technicalities related to measurableselection arguments needed for the proof of the classical dynamic programmingprinciple (3.9) after a convenient relaxation of the control problem see, e.g., ElKaroui and Jeanblanc [16].

Proof of Theorem 3.3. For simplicity, we consider the finite horizon case T < 1,so that, without loss of generality, we assume f D k D 0; see Remark 3.2 (iii). Theextension to the infinite horizon framework is immediate.

1. Let � 2 Ut be arbitrary and set � WD �� . Then

E�g�Xt;x;�

T

� jF�

.!/ D J.�.!/; Xt;x;�

� .!/I Q�!/;

where Q�! is obtained from � by freezing its trajectory up to the stopping time � .Since, by definition, J.�.!/; X

t;x;�� .!/I Q�!/ � V �.�.!/; X

t;x;�� .!//, it follows

from the tower property of conditional expectations that

E�g�X

t;x;�T

� D E�E�g�X

t;x;�T

� jF�

� E�V � ��; X

t;x;��

�;

which provides the second inequality of Theorem 3.3 by the arbitrariness of� 2 Ut .


2. Let " > 0 be given, and consider an arbitrary function

' W S �! R such that ' upper semicontinuous and V � ':

(a) There is a family .�.s;y/;"/.s;y/2S � U0 such that:

�.s;y/;" 2 Us and J.s; yI �.s;y/;"/ � V.s; y/ � "; for every.s; y/ 2 S:

(3.10)

Since g is lower semicontinuous and has quadratic growth, it followsfrom Theorem 3.1 that the function .t 0; x0/ 7! J.t 0; x0I �.s;y/;"/ is lowersemicontinuous, for fixed .s; y/ 2 S. Together with the upper semicontinuityof ', this implies that we may find a family .r.s;y//.s;y/2S of positive scalarsso that, for any .s; y/ 2 S,

'.s; y/ � '.t 0; x0/ � �" and J.s; yI �.s;y/;"/ � J.t 0; x0I �.s;y/;"/ � "

for .t 0; x0/ 2 B.s; yI r.s;y//; (3.11)

where, for r > 0 and .s; y/ 2 S,

B.s; yI r/ WD ˚.t 0; x0/ 2 S W t 0 2 .s � r; s/; jx0 � yj < r

: (3.12)

Note that we do not use here balls of the usual form Br.s; y/. The factthat t 0 � s for .t 0; x0/ 2 B.s; yI r/ will play an important role in Step 3.Clearly,

˚B.s; yI r/ W .s; y/ 2 S; 0 < r � r.s;y/

forms an open covering

of Œ0; T / � Rd . It then follows from the Lindelof covering theorem, see,

e.g., [15] Theorem 6.3 Chap. VIII, that we can find a countable sequence.ti ; xi ; ri /i�1 of elements of S � R, with 0 < ri � r.ti ;xi / for all i � 1, suchthat S � fT g � R

d [ .[i�1B.ti ; xi I ri //. Set A0 WD fT g � Rd , C�1 WD ;,

and define the sequence

AiC1 WD B.tiC1; xiC1I riC1/ n Ci where Ci WD Ci�1 [ Ai ; i � 0:

With this construction, it follows from (3.10) (3.11), together with the factthat V � ', that the countable family .Ai /i�0 satisfies

.�; Xt;x;�� / 2 [i�0Ai P � a.s.; Ai \ Aj D ; for i ¤ j 2 N;

and J.�I �i;"/ � ' � 3" on Ai for i � 1; (3.13)

where �i;" WD �.ti ;xi /;" for i � 1.(b) We now prove the first inequality in Theorem 3.3. We fix � 2 Ut and � 2

T tŒt;T �. Set An WD [0�i�nAi , n � 1. Given � 2 Ut , we define for s 2 Œt; T �:

�";ns WD 1Œt;� �.s/�s C 1.�;T �.s/

�s1.An/c .�; X

t;x;�� / C

nXiD1

1Ai .�; Xt;x;�� /�i;"

s

!:


Notice that f.�; Xt;x;�� / 2 Ai g 2 F t

� , and therefore �";n 2 Ut . By thedefinition of the neighborhood (3.12), notice that � D � ^ ti � ti onf.�; X�

t;x.�// 2 Ai g. Then, it follows from (3.13) that:

E

hg�X

t;x;�";n

T

�jF�

i1An

��; X

t;x;��

� D V�T; X

t;x;�";n

T

�1A0

��; X

t;x;��

�

CnX

iD1

J.�; Xt;x;�� ; �i;"/1Ai

��; Xt;x;�

�

�

�nX

iD0

�'.�; X

t;x;�� 3"

�1Ai

��; X

t;x;��

�D �

'.�; Xt;x;�� / � 3"

�1An

��; Xt;x;�

�

�;

which, by definition of V and the tower property of conditional expectations,implies

V.t; x/ � J.t; x; �";n/

D E

hE

hg�Xt;x;�";n

T

�jF�

ii� E

��'��; X

t;x;��

� � 3"�

1An

��; X

t;x;��

�CE

�g�X

t;x;�T

�1.An/c

��; X

t;x;��

�:

Since g�Xt;x;�

T

� 2 L1, it follows from the dominated convergence theorem

that:

V.t; x/ � �3" C lim infn!1 E

�'.�; X

t;x;�� /1An

��; X

t;x;��

�D �3" C lim

n!1E�'.�; Xt;x;�

� /C1An

��; Xt;x;�

�

�� lim

n!1E�'.�; X

t;x;�� /�1An

��; X

t;x;��

�D �3" C E

�'.�; X

t;x;�� /

;

where the last equality follows from the left-hand side of (3.13) andfrom the monotone convergence theorem, due to the fact that eitherE�'.�; X

t;x;�� /C < 1 or E

�'.�; X

t;x;�� /� < 1. By the arbitrariness

of � 2 Ut and " > 0, this shows that


E�'.�; Xt;x;�

� /

: (3.14)

3. It remains to deduce the first inequality of Theorem 3.3 from (3.14). Fix r > 0. Itfollows from standard arguments, see, e.g., Lemma 3.5 in [34], that we can find a


sequence of continuous functions .'n/n such that 'n � V� � V for all n � 1 andsuch that 'n converges pointwise to V� on Œ0; T � � Br.0/. Set N WD minn�N 'n

for N � 1 and observe that the sequence .N /N is non-decreasing and convergespointwise to V� on Œ0; T � � Br.0/. By (3.14) and the monotone convergencetheorem, we then obtain

V.t; x/ � limN !1E

�N .��; X�

t;x.��// D E

�V�.��; X�

t;x.��//

: ut

3.3 The Dynamic Programming Equation

The dynamic programming equation is the infinitesimal counterpart of the dynamicprogramming principle. It is also widely called the Hamilton-Jacobi-Bellmanequation. In this section, we shall derive it under strong smoothness assumptionson the value function. Let Sd be the set of all d � d symmetric matrices with realcoefficients, and define the map H : S � R � R

d � Sd by

H.t; x; r; p; /

WD supu2U

��k.t; x; u/r C b.t; x; u/ � p C 1

2TrŒ��T.t; x; u/� C f .t; x; u/

�:

We also need to introduce the linear second-order operator Lu associated to thecontrolled process fˇu.0; t/Xu

t ; t � 0g controlled by the constant control process u:

Lu'.t; x/ WD �k.t; x; u/'.t; x/ C b.t; x; u/ � D'.t; x/

C1

2Tr��T.t; x; u/D2'.t; x/

;

where D and D2 denote the gradient and the Hessian operators with respect to thex variable. With this notation, we have by Ito’s formula

ˇ�.0; s/'.s; X�s / � ˇ�.0; t/'.t; X�

t / DZ s

t

ˇ�.0; r/ .@t C L�r / '.r; X�r /dr

CZ s

t

ˇ�.0; r/D'.r; X�r / � �.r; X�

r ; �r /dWr

for every s � t and smooth function ' 2 C 1;2.Œt; s�;Rd / and each admissible controlprocess � 2 U0.

Proposition 3.4. Assume the value function V 2 C 1;2.Œ0; T /;Rd /, and let thecoefficients k.�; �; u/ and f .�; �; u/ be continuous in .t; x/ for all fixed u 2 U

and k bounded from below. Then, V is a classical supersolution of the dynamicprogramming equation, i.e. for all .t; x/ 2 S,

3.3 The Dynamic Programming Equation 31

� @t V .t; x/ � H�t; x; V .t; x/; DV.t; x/; D2V.t; x/

� � 0: (3.15)

Proof. Let .t; x/ 2 S and u 2 U be fixed, and consider the constant control process� D u, together with the associated state process X with initial data Xt D x. For allh > 0, define the stopping time:

�h WD inf fs > t W .s � t; Xs � x/ 62 Œ0; h/ � ˛Bg ;

where ˛ > 0 is some given constant, and B denotes the unit ball of Rd . Notice that�h �! t , P�a.s. when h & 0, and �h D t C h < T for h � Nh.!/ sufficiently small.

1. From the first inequality of the dynamic programming principle, together withthe continuity of V , it follows that

0 � Et;x

"ˇu.0; t/V .t; x/ � ˇu.0; �h/V .�h; X�h

/ �Z �h

t

ˇu.0; r/f .r; Xr; u/dr

#

D �Et;x

"Z �h

t

ˇu.0; r/.@tV C L�V C f /.r; Xr; u/dr

#

�Et;x

"Z �h

t

ˇu.0; r/DV.r; Xr/ � �.r; Xr ; u/dWr

#I

the last equality follows from Ito’s formula and uses the crucial smoothnessassumption on V .

2. Observe that ˇ.0; r/DV.r; Xr/ � �.r; Xr ; u/ is bounded on the stochastic intervalŒt; �h�. Therefore, the second expectation on the right hand-side of the lastinequality vanishes, and we obtain:

�Et;x

"1

h

Z �h

t

ˇu.0; r/.@t V C L�V C f /.r; Xr ; u/dr

#� 0

We now send h to zero. The a.s. convergence of the random value inside theexpectation is easily obtained by the mean value theorem; recall that �h D h

for sufficiently small h > 0. Since the random variable h�1R �h

t ˇu.0; r/.L�V Cf /.r; Xr; u/dr is essentially bounded, uniformly in h, on the stochastic intervalŒt; �h�, it follows from the dominated convergence theorem that

�@t V .t; x/ � LuV.t; x/ � f .t; x; u/ � 0:

By the arbitrariness of u 2 U , this provides the required claim. utWe next wish to show that V satisfies the nonlinear partial differential equa-

tion (3.15) with equality. This is a more technical result which can be proved bydifferent methods. We shall report a proof, based on a contradiction argument, whichprovides more intuition on this result, although it might be slightly longer than theusual proof reported in standard textbooks.


Proposition 3.5. Assume V 2 C 1;2.Œ0; T /;Rd / and H.:; V:DV; D2V / < 1.Assume further that k is bounded and the function H is upper semicontinuous.Then, V is a classical subsolution of the dynamic programming equation, i.e. for all.t; x/ 2 S


� � 0: (3.16)

Proof. Let .t0; x0/ 2 Œ0; T / � Rd be fixed, assume to the contrary that

@t V .t0; x0/ C H�t0; x0; V .t0; x0/; DV.t0; x0/; D2V.t0; x0/

�< 0; (3.17)

and let us work towards a contradiction.

1. For a given parameter " > 0, define the smooth function ' � V by

'.t; x/ WD V.t; x/ C "�jt � t0j2 C jx � x0j4� :

Then

.V � '/.t0; x0/ D 0; .DV � D'/.t0; x0/ D 0; .@t V � @t '/.t0; x0/ D 0;

and .D2V � D2'/.t0; x0/ D 0;

and it follows from the continuity of H and (3.17) that:

h.t; x/ WD �@t ' C H.:; '; D'; D2'/

�.t; x/ < 0 for .t; x/ 2 Nr ; (3.18)

for some sufficiently small parameter r > 0, where Nr WD .t0 � r; t0 C r/ �rB.x0/ � Œ�r; T � � R

n, and B.x0/ is the open unit ball centered at x0.2. From the definition of ', we have

� � WD max@Nr

.V � '/ < 0: (3.19)

For an arbitrary control process � 2 Ut0 , we define the stopping time

�� WD infft > t0 W �t; Xt0;x0;�t / 62 Nrg;

and we observe that��; X

t0;x0;��

� 2 @Nr by the pathwise continuity of thecontrolled process. Then, it follows from (3.19) that

'��; X

t0;x0;��

� � � C V��; X

t0;x0;��

�: (3.20)

3. For notation simplicity, we set ˇ�s WD ˇ�.t0; s/. Since ˇ�

t0D 1, it follows

from Ito’s formula together with the boundedness of ˇ (implied by the lowerboundedness of k) that:

3.4 On the Regularity of the Value Function 33

V.t0; x0/ D '.t0; x0/

D E

hˇ�

�� '��; X

t0;x0;��

� �Z ��

t0

ˇ�s

�@t C L�s

�'�s; Xt0;x0;�

s

�dsi

� E

hˇ�

�� '��; X

t0;x0;��

�CZ ��

t0

ˇ�s

�f .:; �s/ � h

��s; Xt0;x0;�

s

�dsi

by the definition of h. Since .s; Xt0;x0;�s / 2 Nr on Œt0; ��/, it follows from (3.18)

and (3.20) that

V.t0; x0/ � �E�ˇ�

��

C E

"Z ��

t0

ˇ�s f

�s; Xt0;x0;�

s ; �s

�ds C ˇ�

�� V��; X

t0;x0;��

�#

� �e�r jkCj1 C E

"Z ��

t0

ˇ�s f

�s; Xt0;x0;�

s ; �s

�ds C ˇ�

�� V��; X

t0;x0;��

�#:

Since � > 0 does not depend on �, it follows from the arbitrariness of � 2 Ut0

and the continuity of V that the last inequality is in contradiction with the secondinequality of the dynamic programming principle of Theorem 3.3. utAs a consequence of Propositions 3.4 and 3.5, we have the main result of this

section:

Theorem 3.6. Let the conditions of Propositions 3.5 and 3.4 hold. Then, the valuefunction V solves the Hamilton-Jacobi-Bellman equation

� @t V � H�:; V; DV; D2V

� D 0 on S: (3.21)

3.4 On the Regularity of the Value Function

The purpose of this paragraph is to show that the value function should not beexpected to be smooth in general. We start by proving the continuity of the valuefunction under strong conditions; in particular, we require the set U in whichthe controls take values to be bounded. We then give a simple example in thedeterministic framework where the value function is not smooth. Since it is wellknown that stochastic problems are “more regular” than deterministic ones, we alsogive an example of stochastic control problem whose value function is not smooth.

3.4.1 Continuity of the Value Function for Bounded Controls

For simplicity, we reduce the stochastic control problem to the case f D k � 0,see Remark 3.2iii). We will also concentrate on the finite horizon case T < 1.


The corresponding results in the infinite horizon case can be obtained by similararguments. Our main concern, in this section, is to show the standard argument forproving the continuity of the value function. Therefore, the following results assumestrong conditions on the coefficients of the model in order to simplify the proofs. Wefirst start by examining the value function V.t; �/ for fixed t 2 Œ0; T �.

Proposition 3.7. Let f D k � 0, T < 1, and assume that g is Lipschitzcontinuous. Then

(i) V is Lipschitz in x, uniformly in t .(ii) Assume further that U is bounded. Then V is 1

2�Holder continuous in t , and

there is a constant C > 0 such thatˇV.t; x/ � V.t 0; x/

ˇ � C.1 C jxj/p

jt � t 0jI t; t 0 2 Œ0; T �; x 2 Rd :

Proof. (i) For x; x0 2 Rd and t 2 Œ0; T /, we first estimate that

ˇV.t; x/ � V.t; x0/

ˇ � sup�2U0

E

ˇg�Xt;x;�

T

� � g�Xt;x0;�

T

�ˇ

� Const sup�2U0

E

ˇXt;x;�

T � Xt;x0;�T

ˇ� Const jx � x0j;

where we used the Lipschitz continuity of g together with the flow estimatesof Theorem 2.4, and the fact that the coefficients b and � are Lipschitz in x

uniformly in .t; u/. This completes the proof of the Lipschitz property of thevalue function V .

(ii) To prove the Holder continuity in t , we shall use the dynamic programmingprinciple.

(1) We first make the following important observation. A careful review ofthe proof of Theorem 3.3 reveals that, whenever the stopping times ��

are constant (i.e., deterministic), the dynamic programming principle holdstrue with the semicontinuous envelopes taken only with respect to thex�variable. Since V was shown to be continuous in the first part of thisproof, we deduce that

V.t; x/ D sup�2U0

E�V�t 0; Xt;x;�

t 0

�(3.22)

for all x 2 Rd , t < t 0 2 Œ0; T �.

(2) Fix x 2 Rd , t < t 0 2 Œ0; T �. By the dynamic programming principle (3.22),

we have


jV.t; x/ � V.t 0; x/j Dˇˇ sup�2U0

E�V�t 0; Xt;x;�

t 0

� � V.t 0; x/

ˇˇ

� sup�2U0

EˇV�t 0; X

t;x;�t 0

� � V.t 0; x/ˇ:

By the Lipschitz continuity of V.s; �/ established in the first part of thisproof, we see that

jV.t; x/ � V.t 0; x/j � Const sup�2U0

EˇXt;x;�

t 0 � xˇ: (3.23)

We shall now prove that

sup�2U

EˇX

t;x;�t 0 � x

ˇ � Const .1 C jxj/jt � t 0j1=2; (3.24)

which provides the required .1=2/�Holder continuity in view of (3.23).By definition of the process X , and assuming t < t 0, we have

EˇXt;x;�

t 0 � xˇ2 D E

ˇˇZ t 0

t

b.r; Xr; �r /dr CZ t 0

t

�.r; Xr ; �r /dWr

ˇˇ2

� Const E

"Z t 0

t

jh.r; Xr; �r /j2 dr

#

where h WD Œb2 C �2�1=2. Since h is Lipschitz continuous in .t; x; u/ andhas quadratic growth in x and u, this provides

EˇX

t;x;�t 0 � x

ˇ2 � Const

Z t 0

t

.1 C jxj2 C j�r j2/dr

CZ t 0

t

EˇXt;x;�

r � xˇ2

dr

!:

Since the control process � is uniformly bounded, we obtain by theGronwall lemma the estimate:

EˇX

t;x;�t 0 � x

ˇ2 � Const .1 C jxj/jt 0 � t j; (3.25)

where the constant does not depend on the control �. utRemark 3.8. When f and/or k are non-zero, the conditions required on f and k inorder to obtain the .1=2/�Holder continuity of the value function can be deducedfrom the reduction of Remark 3.2(iii).


Remark 3.9. Further regularity results can be proved for the value function underconvenient conditions. Typically, one can prove that LuV exists in the generalizedsense, for all u 2 U . This implies immediately that the result of Proposition 3.5holds in the generalized sense. More technicalities are needed in order to derive theresult of Proposition 3.4 in the generalized sense. We refer to [20], Sect. IV.10, fora discussion of this issue and to Krylov [27] for the technical proofs.

3.4.2 A Deterministic Control Problem with Non-smoothValue Function

Let � � 0; b.x; u/ D u; U D Œ�1; 1� , and n D 1. The controlled state is thenthe one-dimensional deterministic process defined by

Xs D Xt CZ s

t

�t dt for 0 � t � s � T :

Consider the deterministic control problem

V.t; x/ WD sup�2U

.XT /2:

The dynamic programming equation corresponding to this problem is:

�@t V � jD�j D 0:

The value function of this problem is easily seen to be given by

V.t; x/ D�

.x C T � t/2 for x � 0 with optimal control Ou D 1;

.x � T C t/2 for x � 0 with optimal control Ou D �1:

This function is continuous. However, a direct computation shows that it is notdifferentiable at x D 0.

3.4.3 A Stochastic Control Problem with Non-smoothValue Function

Let U D R and the controlled process X� be the scalar process defined by thedynamics:

dX�t D �t dWt;


where W is a scalar Brownian motion. Then, for any � 2 U0, the process X� is amartingale. Let g be a function defined on R with linear growth jg.x/j � C.1Cjxj/for some constant C > 0. Then g.X�

T / is integrable for all T � 0. Consider thestochastic control problem


Et;x

�g.X�

T /

:

Let us assume that V is smooth, and work towards a contradiction.

1. If V is C 1;2.Œ0; T /;R/, then it follows from Proposition 3.4 that V is asupersolution of the dynamic programming equation:

�@t V � 1

2u2D2V � 0 for all u 2 R;

and all .t; x/ 2 Œ0; T / � R. By sending u to infinity, it follows that

V.t; �/ is concave for all t 2 Œ0; T /: (3.26)

2. Notice that V.t; x/ � Et;x

�g.X0

T / D g.x/. Then, it follows from (3.26) that:

V.t; x/ � gconc.x/ for all .t; x/ 2 Œ0; T / � R; (3.27)

where gconc denotes the concave envelope of g, i.e., the smallest concavemajorant of g. If gconc D 1, this already proves that V D 1. We then continuein the case that gconc < 1.

3. Since g � gconc, we see that


Et;x

�g.X�

T / � sup

�2U0

Et;x

�gconc.X�

T /

: � gconc.x/;

where the last equality follows from the Jensen inequality together with themartingale property of the controlled process X� . In view of (3.27), we havethen proved that

V 2 C 1;2.Œ0; T /;R/

H) V.t; x/ D gconc.x/ for all .t; x/ 2 Œ0; T / � R:

Now recall that this implication holds for any arbitrary function g with lineargrowth. We then obtain a contradiction whenever the function gconc is not C 2.R/.Hence,

gconc 62 C 2.R/ H) V 62 C 1;2.Œ0; T /;R2/:

Chapter 4Optimal Stopping and Dynamic Programming

As in the previous chapter, we assume here that the filtration F is defined as theP�augmentation of the canonical filtration of the Brownian motion W defined onthe probability space .�;F ;P/.

Our objective is to derive similar results, as those obtained in the previouschapter for standard stochastic control problems, in the context of optimal stoppingproblems. We will then first start with the formulation of optimal stopping problems,then the corresponding dynamic programming principle, and dynamic programmingequation.

4.1 Optimal Stopping Problems

For 0 � t � T < 1, we denote by TŒt;T � the collection of all F�stopping timeswith values in Œt; T �. We also recall the notation S WD Œ0; T / � R

n for the parabolicstate space of the underlying state process X defined by the stochastic differentialequation:

dXt D b.t; Xt/dt C �.t; Xt /dWt; (4.1)

where b and � are defined on NS and take values in Rn and Sn, respectively. We

assume that b and � satisfy the usual Lipschitz and linear growth conditions so thatthe above SDE has a unique strong solution satisfying the integrability proved inTheorem 2.2.

The infinitesimal generator of the Markovian diffusion process X is denoted by

A' WD b �D' C 1

2Tr��TD2'

�:


39

40 4 Optimal Stopping and Dynamic Programming

Let g be a continuous function from Rn to R and assume that

E

"

sup0�t�T

jg.Xt /j#

< 1: (4.2)

For instance, if g has linear growth, the previous integrability condition is automat-ically satisfied. Under this condition, the criterion

J.t; x; �/ WD E�g�Xt;x�

��(4.3)

is well defined for all .t; x/ 2 S and � 2 TŒt;T �. Here, Xt;x denotes the unique strongsolution of (4.1) with initial conditionXt;x

t D x.The optimal stopping problem is now defined by:

V.t; x/ WD sup�2TŒt;T �

J.t; x; �/ for all .t; x/ 2 S: (4.4)

A stopping time O� 2 TŒt;T � is called an optimal stopping rule if V.t; x/ D J.t; x; O� /.The set

S WD f.t; x/ W V.t; x/ D g.x/g (4.5)

is called the stopping region and is of particular interest: whenever the state isin this region, it is optimal to stop immediately. Its complement Sc is called thecontinuation region.

Remark 4.1. As in the previous chapter we could have allowed for the infinitehorizon T � 1, and we could have considered the apparently more generalcriterion

NV .t; x/ WD sup�2TŒt;T �

E

�Z �

t

ˇ.t; s/f .s; Xs/ds C ˇ.t; �/g�Xt;x�

�1�<1

�;

with

ˇ.t; s/ WD e� R st k.s;Xs/ds for 0 � t � s < T:

In the finite horizon case, this problem may be reduce to the context of (4.4) byintroducing the additional states

Yt WD Y0 CZ t

0

ˇsf .s; Xs/ds;

Zt WD Z0 CZ t

0

Zsk.s; Xs/ds:

Also, the methodology which will be developed for the problem (4.4) naturallyextends to the infinite horizon case.


Remark 4.2. Consider the subset of stopping rules:

T tŒt;T � WD ˚

� 2 TŒt;T � W � independent of Ft�: (4.6)

By a similar argument as in Remark 3.2iv), we can see that the maximization in theoptimal stopping problem (4.4) can be restricted to this subset, i.e.,

V.t; x/ WD sup�2T t

Œt;T �

J.t; x; �/ for all .t; x/ 2 S: (4.7)

4.2 The Dynamic Programming Principle

The proof of the dynamic programming principle for optimal stopping problems iseasier than in the context of stochastic control problems of the previous chapter. Thereader may consult the excellent exposition in the book of Karatzas and Shreve [24],Appendix D, where the following dynamic programming principle is proved:

V.t; x/ D sup�2T t

Œt;T �

E�1f�<�gg.Xt;x

� /C 1f��gV.�;Xt;x� /�; (4.8)

for all .t; x/ 2 S and � 2 TŒt;T �. In particular, the proof in the previous referencedoes not require any heavy measurable selection, and is essentially based on thesupermartingale nature of the so-called Snell envelope process. Moreover, weobserve that it does not require any Markovian property of the underlying stateprocess.

We report here a different proof in the spirit of the weak dynamic program-ming principle for stochastic control problems proved in the previous chapter.The subsequent argument is specific to our Markovian framework and, in thissense, is weaker than the classical dynamic programming principle. However, thecombination of the arguments of this chapter with those of the previous chapterallows to derive a dynamic programming principle for mixed stochastic control andstopping problems.

The following claim will make use of the subset T tŒt;T �, introduced in (4.6), of all

stopping times in TŒt;T � which are independent of Ft , and the notations:



V .t 0; x0/;

for all .t; x/ 2 NS. We recall that V� and V � are the lower-and upper-semicontinuousenvelopes of V , and that V� D V � D V whenever V is continuous.

Theorem 4.3. Assume that V is locally bounded. For .t; x/ 2 S, let � 2 T tŒt;T � be a

stopping time such that Xt;x� is L1.P/� bounded. Then


V.t; x/ � sup�2T t

Œt;T �


� /C 1f��gV �.�;Xt;x� /�; (4.9)

V.t; x/ � sup�2T t

Œt;T �


� /C 1f��gV�.�;Xt;x� //

�: (4.10)

Proof. Inequality (4.9) follows immediately from the tower property and the factthat J � V �.

We next prove inequality (4.10) with V� replaced by an arbitrary function

' W S �! R such that ' is upper-semicontinuous and V � ';

which implies (4.10) by the same argument as in Step 3 of the proof of Theorem 3.3.Arguing as in Step 2 of the proof of Theorem 3.3, we first observe that, for every

" > 0, we can find a countable family NAi � .ti � ri ; ti � � Ai � S, together with asequence of stopping times �i;" in T ti

Œti ;T �, i � 1, satisfying NA0 D fT g � R

d and

[i�0 NAi D S; NAi \ NAj D ; for i ¤ j 2 N; NJ .�I �i;"/ � ' � 3" on NAi for i � 1:

(4.11)

Set NAn WD [i�n NAi , n � 1. Given two stopping times �; � 2 T tŒt;T �, it is easily

checked that

�n;" WD �1f�<�g C 1f��g

T 1. NAn/c��;Xt;x

�

�CnX

iD1� i;"1 NAi

��;Xt;x

�

�!

defines a stopping time in T tŒt;T �. We then deduce from the tower property and (4.11)

that

NV .t; x/ � NJ .t; xI �n;"/� E

�g�Xt;x�

�1f�<�g C 1f��g

�'.�;X

t;x� /� 3"

�1 NAn.�;X

t;x� /�

CE

h1f��gg.Xt;x

T /1. NAn/c .�; Xt;x� /i:

By sending n ! 1 and arguing as in the end of Step 2 of the proof of Theorem 3.3,we deduce that

NV .t; x/ � E�g�Xt;x�

�1f�<�g C 1f��g'.�;Xt;x

� /� � 3";

and the result follows from the arbitrariness of " > 0 and � 2 T tŒt;T �. ut

Remark 4.4. In the context of the optimal stopping NV introduced in Remark 4.1,denoting ˇst WDˇ.t; s/, the dynamic programming principle of Theorem 4.3translates to

4.3 The Dynamic Programming Equation 43

NV .t; x/ � sup�2T t

Œt;T �

E

"Z �^�

t

ˇst f .s;Xt;xs /ds C ˇ�^�

t

1�<�g.Xt;x

� /C 1��NV �.�;X

t;x� /

#

;

NV .t; x/ � sup�2T t

Œt;T �

E

"Z �^�

t

ˇst f .s;Xt;xs /ds C ˇ�^�

t

1�<�g.Xt;x

� /C 1��NV�.�;X

t;x�/#

;

and suitable conditions inherited from the reduction of Remark 4.1. In this setting,the derivation of the dynamic programming equation, in the subsequent sections,involves the linear second order differential operator

L' WD �k' C A' D �k' C b �D' C 1

2Tr��TD2'

�: (4.12)

4.3 The Dynamic Programming Equation

In this section, we explore the infinitesimal counterpart of the dynamic program-ming principle of Theorem 4.3, when the value function V is a priori known tobe smooth. The smoothness that will be required in this chapter must be so thatIto’s formula applies to V . In particular, V is continuous, and the dynamic pro-gramming principle of Theorem 4.3 reduces to the classical dynamic programmingprinciple (4.8).

Loosely speaking, the following dynamic programming equation says the fol-lowing:

• In the stopping region S defined in (4.5), continuation is sub-optimal, andtherefore the linear PDE must hold with inequality in such a way that the valuefunction is a submartingale.

• In the continuation region Sc , it is optimal to delay the stopping decision aftersome small moment, and therefore the value function must solve a linear PDE asin Chap. 2.

Theorem 4.5. Assume that V 2 C1;2 .Œ0; T /;Rn/, and let g W Rn �! R be

continuous. Then V solves the obstacle problem:

min f�.@t C A/V; V � gg D 0 on S: (4.13)

Proof. We organize the proof into two steps.

1. We first show the classical subsolution property:

min f�.@t C A/V; V � gg � 0 on S: (4.14)


The inequality V � g � 0 is obvious as the constant stopping rule � D t 2 TŒt;T �is admissible. Next, for .t0; x0/ 2 S, consider the stopping times

�h WD inf˚t > t0 W .t; Xt0;x0

t / 62 Œt0; t0 C h� � B� ; h > 0;

where B is the unit ball of Rn centered at x0. Then �h 2 T tŒt;T / for sufficiently

small h, and it follows from (4.10) and the continuity of V that

V.t0; x0/ � E�V��h;X�h

��:

We next apply Ito’s formula, and observe that the expected value of the diffusionterm vanishes because .t; Xt/ lies in the compact subset Œt0; t0 C h� � B fort 2 Œt0; �h�. Then

E

"�1h

Z �h

t0

.@t C A/V .t; Xt0;x0t /dt

#

� 0:

Clearly, there exists Oh! > 0, depending on!, �h D h for h � Oh! . Then, it followsfrom the mean value theorem that the expression inside the expectation convergesP�a.s. to �.@t C A/V .t0; x0/, and we conclude by dominated convergence that�.@t C A/V .t0; x0/ � 0.

2. In order to complete the proof, we use a contradiction argument, assuming that

V.t0; x0/ > g.t0; x0/ and � .@t C A/V .t0; x0/ > 0 at some .t0; x0/ 2 S (4.15)

and we work towards a contradiction of (4.9). Introduce the function

'.t; x/ WD V.t; x/C ".jx � x0j4 C jt � t0j2/ for .t; x/ 2 S:

Then, it follows from (4.15) that for a sufficiently small " > 0, we may findh > 0 and ı > 0 such that

V � g C ı and � .@t C A/' � 0 on Nh WD Œt0; t0 C h� � hB: (4.16)

Moreover,

� � WD max@Nh

.V � '/ < 0: (4.17)

Next, let

� WD inf˚t > t0 W �

t; Xt0;x0t

� 62 Nh

�:

For an arbitrary stopping rule � 2 T tŒt;T �, we compute by Ito’s formula that

4.4 Regularity of the Value Function 45

E ŒV .� ^ �;X�^� / � V.t0; x0/� D E Œ.V � '/ .� ^ �;X�^� /�

CE Œ' .� ^ �;X�^� /� '.t0; x0/�

D E Œ.V � '/ .� ^ �;X�^� /�

CE

"Z �^�

t0

.@t C A/'.t; Xt0;x0t /dt

#

;

where the diffusion term has zero expectation because the process .t; Xt0;x0t / is

confined to the compact subset Nh on the stochastic interval Œt0; � ^ ��. Since�.@t C A/' � 0 on Nh by (4.16), this provides

E ŒV .� ^ �;X�^� /� V.t0; x0/� � E Œ.V � '/ .� ^ �;X�^� /�

� ��PŒ� � ��;

by (4.17). Then, since V � g C ı on Nh by (4.16)

V.t0; x0/ � �PŒ� � ��C E��g.Xt0;x0

� /C ı�

1f�<�g C V��;X

t0;x0�

�1f��g

�

� .� ^ ı/C E�g.Xt0;x0

� /1f�<�g C V��;X

t0;x0�

�1f��g

�:

By the arbitrariness of � 2 T tŒt;T � and the continuity of V , this provides the desired

contradiction of (4.9). utRemark 4.6. In the context of the optimal stopping NV introduced in Remark 4.1,we may derive the dynamic programming equation as the infinitesimal counterpartof the dynamic programming principle of Remark 4.4. Following the same line ofargument as in the previous proof, it follows that for NV 2 C1;2.Œ0; T /;Rn/, thedynamic programming equation is given by the obstacle problem:

min f�@tV � LV � f; V � gg D 0 on S: (4.18)

4.4 Regularity of the Value Function

4.4.1 Finite Horizon Optimal Stopping

In this subsection, we consider the case T < 1. Similar to the continuity result ofProposition 3.7 for the stochastic control framework, the following continuity resultis obtained as a consequence of the flow continuity of Theorem 2.4 together withthe dynamic programming principle.


Proposition 4.7. Assume g is Lipschitz continuous, and let T < 1. Then, there isa constant C such that

ˇˇV.t; x/ � V.t 0; x0/

ˇˇ � C

jx � x0j C

pjt � t 0j

for all .t; x/; .t 0; x0/ 2 S:

Proof. (i) For t 2 Œ0; T � and x; x0 2 Rn, it follows from the Lipschitz property of

g that

ˇˇV.t; x/ � V.t; x0/

ˇˇ � Const sup

�2TŒt;T �E

ˇˇˇXt;x

� � Xt;x0

�

ˇˇˇ

� Const E supt�s�T

ˇˇˇXt;x

� �Xt;x0

�

ˇˇˇ

� Const jx � x0j

by the flow continuity result of Theorem 2.4.(ii) To prove the Holder continuity result in t , we argue as in the proof of

Proposition 3.7 using the dynamic programming principle of Theorem 4.3.

(1) We first observe that, whenever the stopping time � D t 0 > t is constant(i.e. deterministic), the dynamic programming principle (4.9) and (4.10)holds true if the semicontinuous envelopes are taken with respect to thevariable x, with fixed time variable. Since V is continuous in x by the firstpart of this proof, we deduce that

V.t; x/ D sup�2T t

Œt;T �

E�1f�<t 0gg

�Xt;x�

�C 1f��t 0gV�t 0; Xt;x

t 0

��: (4.19)

(2) We then estimate that

0 � V.t; x/ � E�V�t 0; Xt;x

t 0

��

� sup�2T t

Œt;T �

E�1f�<t 0g

�g�Xt;x�

�� V�t 0; Xt;x

t 0

��

� sup�2T t

Œt;T �

E�1f�<t 0g

�g�Xt;x�

�� g�Xt;xt 0

��;

where the last inequality follows from the fact that V � g. Using theLipschitz property of g, this provides:

0 � V.t; x/ � E�V�t 0; Xt;x

t 0

�� Const E

"

supt�s�t 0

ˇXt;xs � Xt;x

t 0

ˇ#

� Const .1C jxj/pt 0 � t


by the flow continuity result of Theorem 2.4. Using this estimate togetherwith the Lipschitz property proved in (i) above, this provides:

jV.t; x/ � V.t 0; x/j � ˇˇV.t; x/ � E

�V�t 0; Xt;x

t 0

��ˇˇ

C ˇE�V�t 0; Xt;x

t 0

�� V.t 0; x/ˇ

� Const.1C jxj/pt 0 � t C E

ˇˇXt;x

t 0 � xˇˇ

� Const .1C jxj/pt 0 � t ;

by using again Theorem 2.4. ut

4.4.2 Infinite Horizon Optimal Stopping

In this section, the state process X is defined by a homogeneous scalar diffusion:

dXt D b.Xt/dt C �.Xt /dWt: (4.20)

We introduce the hitting times:

Hxb WD inf

˚t > 0 W X0;x D b

�;

and we assume that the processX is regular, i.e.,

P�Hxb < 1�

> 0 for all x; b 2 R; (4.21)

which means that there is no subinterval of R from which the processX cannot exit.We consider the infinite horizon optimal stopping problem:

V.x/ WD sup�2T

E�e�ˇ�g

�X0;x�

�1f�<1g

�; (4.22)

where T WD TŒ0;1�, and ˇ > 0 is the discount rate parameter.According to Theorem 4.3, the dynamic programming equation corresponding

to this optimal stopping problem is the obstacle problem:

min fˇv � Av; v � gg D 0;

where the differential operator in the present homogeneous context is given by thegenerator of the diffusion:

Av WD bv0 C 1

2�2v00: (4.23)


The ordinary differential equation

Av � ˇv D 0 (4.24)

has two positive linearly independent solutions

; � � 0 such that ; strictly increasing � strictly decreasing. (4.25)

Clearly and � are uniquely determined up to a positive constant, and all othersolution of (4.24) can be expressed as a linear combination of and �.

The following result follows from an immediate application of Ito’s formula.

Lemma 4.8. For any b1 < b2, we have

E

he�ˇHx

b1 1fHxb1

�Hxb2

gi

D .x/�.b2/� .b2/�.x/

.b1/�.b2/� .b2/�.b1/;

E

he�ˇHx

b2 1fHxb1

�Hxb2

gi

D .b1/�.x/� .x/�.b1/

.b1/�.b2/� .b2/�.b1/:

We now show that the value function V is concave up to some change of variableand provide conditions under which V is C1 across the exercise boundary, i.e., theboundary between the exercise and the continuation regions.

For the next result, we observe that the function . =�/ is continuous and strictlyincreasing by (4.25), and therefore invertible. To prepare for its statement, let usobserve that, on any compact subset of the continuation region, we have V D A CB for some constants A;B 2 R. Then, .V=�/ ı . =�/�1.x/ D Ax C b is affineon any compact subset of the continuation region. The following result examinesthe nature of .V=�/ ı . =�/�1 on the entire domain.

Theorem 4.9. (i) The function .V=�/ ı . =�/�1 is concave. In particular, V iscontinuous on R.

(ii) Let x0 be such that V.x0/ D g.x0/, and assume that g, and � aredifferentiable at x0. Then V is differentiable at x0, and V 0.x0/ D g0.x0/.

Proof. Denote F WD =�. For (i), it is sufficient to prove that:

V�.x/ � V

�.b1/

F.x/ � F.b1/ �V�.b2/ � V

�.x/

F.b2/ � F.x/ for all b1 < x < b2: (4.26)

For " > 0, consider the "�optimal stopping rules �1; �2 2 T for the problems V.b1/and V.b2/:

E�e�ˇ�i g

�X0;x�i

�� V.bi /� " for i D 1; 2:

We next define the stopping time

�" WDHxb1

C �1 ı �Hxb1

1fHx

b1<Hx

b2g C

Hxb2

C �2 ı �Hxb2

1fHx

b2<Hx

b1g;


where � denotes the shift operator on the canonical space. In words, the stoppingrule �" uses the "�optimal stopping rule �1 if the level b1 is reached before the levelb2 and the "�optimal stopping rule �2 otherwise. Then, it follows from the strongMarkovian property that

V.x/ � E

he�ˇ�"g

X0;x�"

i

D E

he�ˇHx

b1E�e�ˇ�1g

�X0;b1�1

��1fHx

b1<Hx

b2gi

CE

he�ˇHx

b21E�e�ˇ�2g

�X0;b2�2

��1fHx

b2<Hx

b1gi

� .V .b1/ � "/Ehe�ˇHx

b11fHxb1<Hx

b2gi

C .V .b2/ � "/Ehe�ˇHx

b21fHxb2<Hx

b1gi:

Sending " & 0, this provides

V.x/ � V.b1/Ehe�ˇHx

b11fHxb1<Hx

b2gi

C V.b2/Ehe�ˇHx

b21fHxb2<Hx

b1gi:

By using the explicit expressions of Lemma 4.8 above, this provides

V.x/

�.x/� V.b1/

�.b1/

F.b2/� F.x/

F.b2/� F.b1/C V.b2/

�.b2/

F.x/ � F.b1/F.b2/ � F.b1/ ;

which implies (4.26).(ii) We next prove the smoothfit result. Let x0 be such that V.x0/ D g.x0/. Then,

since V � g, is strictly increasing and � � 0 is strictly decreasing, it followsfrom (4.26) that:

g

�.x0 C "/� g

�.x0/

�.x0 C "/�

�.x0/

�V�.x0 C "/� V

�.x0/

�.x0 C "/�

�.x0/

�V�.x0 � ı/ � V

�.x0/

�.x0 � ı/ �

�.x0/

�g

�.x0 � ı/� g

�.x0/

�.x0 � ı/�

�.x0/

; (4.27)

for all " > 0, ı > 0. Multiplying by .. =�/.x0 C "/ � . =�/.x0//=", this impliesthat:

g

�.x0 C "/� g

�.x0/

"�

V�.x0 C "/� V

�.x0/

"

� C."/�.ı/

g

�.x0 � ı/ � g

�.x0/

ı; (4.28)


where

C."/ WD

�.x0 C "/�

�.x0/

"and �.ı/ WD

�.x0 � ı/ �

�.x0/

ı:

We next consider two cases:

• If . =�/0.x0/ ¤ 0, then we may take " D ı and send " & 0 in (4.28) to obtain:

dC. V�/

dx.x0/ D

�g

�

�0.x0/: (4.29)

• If . =�/0.x0/ D 0, then, we use the fact that for every sequence "n & 0, there isa subsequence "nk & 0 and ık & 0 such that C."nk / D �.ık/. Then (4.28)reduces to:

g

�.x0 C "nk /� g

�.x0/

"nk�

V�.x0 C "nk / � V

�.x0/

"nk�

g

�.x0 � ık/ � g

�.x0/

ık;

and therefore

V�.x0 C "nk /� V

�.x0/

"nk�!

�g

�

�0.x0/:

By the arbitrariness of the sequence ."n/n, this provides (4.29).

Similarly, multiplying (4.27) by .. =�/.x0/ � . =�/.x0 � ı//=ı, and arguing asabove, we obtain:

d�. V�/

dx.x0/ D

�g

�

�0.x0/;

thus completing the proof. ut

4.4.3 An Optimal Stopping Problem with Nonsmooth Value

We consider the example

Xt;xs WD x C .Wt �Ws/ for s � t:

Let g W R �! RC be a measurable nonnegative function and consider the infinitehorizon optimal stopping problem:


V.t; x/ WD sup�2TŒt;1�

E�g�Xt;x�

�1f�<1g

�

D sup�2TŒt;1/

E�g�Xt;x�

��;

where the last equality follows from the nonnegativity of g together with Fatou’sLemma. Let us assume that V 2 C1;2.S/ and work towards a contradiction. We firstobserve by the homogeneity of the problem that V.t; x/ D V.x/ is independent oft . Moreover, it follows from Theorem 4.5 that V is concave in x and V � g. Then

V � gconc; (4.30)

where gconc is the concave envelope of g. If gconc D 1, then V D 1. We thencontinue in the more interesting case where gconc < 1.

By the Jensen inequality and the non-negativity of g, the process˚g�Xt;xs

�;

s � t�

is a supermartingale, and

V.t; x/ � sup�2TŒt;T �

E�gconc

�Xt;x�

�� gconc.x/:

Hence, V D gconc, and we obtain the required contradiction whenever gconc is notdifferentiable at some point of R.

Chapter 5Solving Control Problems by Verification

In this chapter, we present a general argument, based on Ito’s formula, which allowsto show that some “guess” of the value function is indeed equal to the unknownvalue function. Namely, given a smooth solution v of the dynamic programmingequation, we give sufficient conditions which allow to conclude that v coincideswith the value function V . This is the so-called verification argument. The statementof this result is heavy, but its proof is simple and relies essentially on Ito’s formula.However, depending on the problem in hand, the verification of the conditions whichmust be satisfied by the candidate solution can be difficult.

The verification argument will be provided in the contexts of stochastic controland optimal stopping problems. We conclude this chapter with some examples.

5.1 The Verification Argument for Stochastic ControlProblems

We recall the stochastic control problem formulation of Sect. 3.1. The set of admis-sible control processes U0 � U is the collection of all progressively measurableprocesses with values in the subset U � R

k . For every admissible control process� 2 U0, the controlled process is defined by the stochastic differential equation:

dX�t D b.t; X�

t ; �t /dt C �.t; X�t ; �t /dWt:

The gain criterion is given by

J.t; x; �/ WD E

�Z T

t

ˇ�.t; s/f .s; Xt;x;�s ; �s/ds C ˇ�.t; T /g.X

t;x;�T /

�;

with

ˇ�.t; s/ WD e� R st k.r;X

t;x;�r ;�r /dr :


53

54 5 Solving Control Problems by Verification

The stochastic control problem is defined by the value function:


J.t; x; �/; for .t; x/ 2 S: (5.1)

We follow the notations of Sect. 3.3. We recall the Hamiltonian H : S�R�Rd �Sd

defined by:

H.t; x; r; p; �/

WD supu2U

��k.t; x; u/r C b.t; x; u/ � p C 1

2TrŒ��T.t; x; u/�� C f .t; x; u/

�;

where b and � satisfy the conditions (3.1)–(3.2), and the coefficients f and k aremeasurable. From the results of the previous section, the dynamic programmingequation corresponding to the stochastic control problem (5.1) is

� @t v � H.:; v; Dv; D2v/ D 0 and v.T; :/ D g: (5.2)

A function v will be called a supersolution (resp. subsolution) of the (5.2) if

�@t v � H.:; v; Dv; D2v/ � (resp. �) 0 and v.T; :/ � (resp. �) g:

The proof of the subsequent result will make use of the following linear secondorder operator:

Lu'.t; x/ WD �k.t; x; u/'.t; x/ C b.t; x; u/ � D'.t; x/

C1

2Tr��T.t; x; u/D2'.t; x/

�;

which corresponds to the process fˇu.0; t/Xut ; t � 0g controlled by the constant

control process u, in the sense that

ˇ�.0; s/'.s; X�s / � ˇ�.0; t/'.t; X�

t / DZ s

t

ˇ�.0; r/ .@t C L�r / '.r; X�r /dr

CZ s

t

ˇ�.0; r/D'.r; X�r / � �.r; X�

r ; �r /dWr;

for every t � s and smooth function ' 2 C 1;2.Œt; s�;Rd / and each admissible controlprocess � 2 U0. The last expression is an immediate application of Ito’s formula.

Theorem 5.1. Let T < 1, and v 2 C 1;2.Œ0; T /;Rd / \ C.Œ0; T � � Rd /. Assume

that kk�k1 < 1 and v and f have quadratic growth, i.e., there is a constant C

such that

jf .t; x; u/j C jv.t; x/j � C.1 C jxj2 C juj/; .t; x; u/ 2 Œ0; T / � Rd � U:

5.1 Verification in Stochastic Control 55

(i) Suppose that v is a supersolution of (5.2). Then v � V on Œ0; T � � Rd .

(ii) Let v be a solution of (5.2), and assume that there exists a minimizer Ou.t; x/ ofu 7�! Luv.t; x/ C f .t; x; u/ such that

• 0 D @t v.t; x/ C LOu.t;x/v.t; x/ C f�t; x; Ou.t; x/

• The stochastic differential equation

dXs D b .s; Xs; Ou.s; Xs// ds C � .s; Xs; Ou.s; Xs// dWs

defines a unique solution X for each given initial data Xt D x

• The process O�s WD Ou.s; Xs/ is a well-defined control process in U0.

Then v D V , and O� is an optimal Markovian control process.

Proof. Let � 2 U0 be an arbitrary control process, X the associated state processwith initial date Xt D x, and define the stopping time

�n WD .T � n�1/ ^ inf fs > t W jXs � xj � ng :

By Ito’s formula, we have

v.t; x/ D ˇ.t; �n/v .�n; X�n/ �Z �n

t

ˇ.t; r/.@t C L�r /v.r; Xr/dr

�Z �n

t

ˇ.t; r/Dv.r; Xr/ � �.r; Xr ; �r/dWr:

Observe that .@t C L�r /v C f .�; �; u/ � @t v C H.�; �; v; Dv; D2v/ � 0, and thatthe integrand in the stochastic integral is bounded on Œt; �n�, a consequence of thecontinuity of Dv, � , and the condition kk�k1 < 1. Then:

v.t; x/ � E

"ˇ.t; �n/v .�n; X�n/ C

Z �n

t

ˇ.t; r/f .r; Xr ; �r /dr

#: (5.3)

We now take the limit as n increases to infinity. Since �n �! T a.s. andˇˇˇ.t; �n/v .�n; X�n/ C

Z �n

t


ˇˇ

� C eT kk�k1

1 C jX�n j2 C T C

Z T

t

jXsj2ds

�

� C eT kk�k1.1 C T /

1 C sup

t�s�T

jXsj2 CZ T

t

j�sj2ds

!2 L

1;


by the estimate (3.5) of Theorem 3.1, it follows from the dominated convergencethat

v.t; x/ � E

�ˇ.t; T /v.T; XT / C

Z T

t


�

� E

�ˇ.t; T /g.XT / C

Z T

t


�;

where the last inequality uses the condition v.T; �/ � g. Since the control � 2 U0 isarbitrary, this completes the proof of (i).

Statement (ii) is proved by repeating the above argument and observing that thecontrol O� achieves equality at the crucial step (5.3). utRemark 5.2. When U is reduced to a singleton, the optimization problem V isdegenerate. In this case, the DPE is linear, and the verification theorem reducesto the so-called Feynman-Kac formula.

Notice that the verification theorem assumes the existence of such a solution, andis by no means an existence result. However, it provides uniqueness in the class offunctions with quadratic growth.

We now state without proof an existence result for the DPE together with theterminal condition V.T; �/ D g (see [23] and the references therein). The mainassumption is the so-called uniform parabolicity condition:

there is a constant c > 0 such that

� 0 �� 0.t; x; u/ � � cj�j2 for all .t; x; u/ 2 Œ0; T � � Rn � U: (5.4)

In the following statement, we denote by C kb .Rn/ the space of bounded functions

whose partial derivatives of orders � k exist and are bounded continuous. Wesimilarly denote by C

p;k

b .Œ0; T �;Rn/ the space of bounded functions whose partialderivatives with respect to t , of orders � p, and with respect to x, of order � k, existand are bounded continuously

Theorem 5.3. Let Condition 5.4 hold, and assume further that:

• U is compact• b, � , and f are in C 1;2

b .Œ0; T �;Rn/

• g 2 C 3b .Rn/

Then the DPE (3.21) with the terminal data V.T; �/ D g has a unique solution V 2C 1;2

b .Œ0; T � � Rn/.

5.2 Examples 57

5.2 Examples of Control Problems with Explicit Solutions

5.2.1 Optimal Portfolio Allocation

We now apply the verification theorem to a classical example in finance, which wasintroduced by Merton [29, 30], and generated a huge literature since then.

Consider a financial market consisting of a nonrisky asset S0 and a risky one S .The dynamics of the price processes is given by

dS0t D S0

t rdt and dSt D St Œ�dt C �dWt � :

Here, r; �, and � are some given positive constants, and W is a one-dimensionalBrownian motion.

The investment policy is defined by an F�adapted process D ft ; t 2 Œ0; T �g,where t represents the amount invested in the risky asset at time t ; The remainingwealth .Xt � t / is invested in the risky asset. Therefore, the liquidation value of aself-financing strategy satisfies

dXt D t

dSt

St

C .Xt � t /

dS0t

S0t

D .rXt C .� � r/t / dt C �t dWt: (5.5)

Such a process is said to be admissible if it lies in U0 D H2 which will be referred

to as the set of all admissible portfolios. Observe that, in view of the particular formof our controlled process X , this definition agrees with (3.4).

Let � be an arbitrary parameter in .0; 1/ and define the power utility function:

U.x/ WD x� for x � 0:

The parameter � is called the relative risk aversion coefficient.The objective of the investor is to choose an allocation of his wealth so as to

maximize the expected utility of his terminal wealth, i.e.,

V.t; x/ WD sup2U0

E�U.X

t;x;T /

�;

where Xt;x; is the solution of (5.5) with initial condition Xt;x;t D x.

The dynamic programming equation corresponding to this problem is

@w

@t.t; x/ C sup

u2RAuw.t; x/ D 0; (5.6)


where Au is the second-order linear operator

Auw.t; x/ WD .rx C .� � r/u/@w

@x.t; x/ C 1

2�2u2 @2w

@x2.t; x/:

We next search for a solution of the dynamic programming equation of the formv.t; x/ D x� h.t/. Plugging this form of solution into the PDE (5.6), we get thefollowing ordinary differential equation on h:

0 D h0 C �h supu2R

�r C .� � r/

u

xC 1

2.� � 1/�2 u2

x2

�(5.7)

D h0 C �h supı2R

�r C .� � r/ı C 1

2.� � 1/�2ı2

�(5.8)

D h0 C �h

�r C 1

2

.� � r/2

.1 � �/�2

�; (5.9)

where the maximizer is

Ou WD � � r

.1 � �/�2x:

Since v.T; �/ D U.x/, we seek for a function h satisfying the above ordinarydifferential equation together with the boundary condition h.T / D 1. This inducesthe unique candidate:

h.t/ WD ea.T �t / with a WD �

�r C 1

2

.� � r/2

.1 � �/�2

�:

Hence, the function .t; x/ 7�! x� h.t/ is a classical solution of the HJB equa-tion (5.6). It is easily checked that the conditions of Theorem 5.1 are all satisfiedin this context. Then V.t; x/ D x�h.t/, and the optimal portfolio allocation policyis given by the linear control process:

Ot D � � r

.1 � �/�2X O

t :

5.2.2 Law of Iterated Logarithm for Double StochasticIntegrals

The main object of this paragraph is Theorem 5.5, reported from [11], whichdescribes the local behavior of double stochastic integrals near the starting pointzero. This result will be needed in the problem of hedging under gamma constraintswhich will be discussed later in these notes. An interesting feature of the proof of

5.2 Examples 59

Theorem 5.5 is that it relies on a verification argument. However, the problem doesnot fit exactly in the setting of Theorem 5.1. Therefore, this is an interesting exerciseon the verification concept.

Given a bounded predictable process b, we define the processes

Y bt WD Y0 C

Z t

0

brdWr and Zbt WD Z0 C

Z t

0

Y br dWr; t � 0 ;

where Y0 and Z0 are some given initial data in R.

Lemma 5.4. Let and T be two positive parameters with 2T < 1. Then:

Ehe2Zb

T

i� E

he2Z1

T

ifor each predictable process b with kbk1 � 1:

Proof. We split the argument into three steps

1. We first directly compute that

Eh

e2Z1T

ˇFt

iD v.t; Y 1

t ; Z1t /;

where, for t 2 Œ0; T �, and y; z 2 R, the function v is given by:

v.t; y; z/ WD E

�exp

2

�z C

Z T

t

.y C Wu � Wt / dWu

��

D e2zE�exp

�˚2yWT �t C W 2

T �t � .T � t/��

D � exp�2z � .T � t/ C 2�22.T � t/y2

�;

where � WD Œ1 � 2.T � t/��1=2. Observe that

the function v is strictly convex in y, (5.10)

and

yD2yzv.t; y; z/ D 8�23.T � t/ v.t; y; z/ y2 � 0: (5.11)

2. For an arbitrary real parameter ˇ, we denote by Aˇ the generator the process�Y b; Zb

:

Aˇ WD 1

2ˇ2D2

yy C 1

2y2D2

zz C ˇyD2yz:

In this step, we intend to prove that for all t 2 Œ0; T � and y; z 2 R,�@t � C max

jˇj�1Aˇv

�.t; y; z/ D �

@t � C A1v.t; y; z/� D 0: (5.12)


The second equality follows from the fact that fv.t; Y 1t ; Z1

t /; t � T g is amartingale . As for the first equality, we see from (5.10) and (5.11) that 1 is amaximizer of both functions ˇ 7�! ˇ2D2

yyv.t; y; z/ and ˇ 7�! ˇyD2yzv.t; y; z/

on Œ�1; 1�.3. Let b be some given predictable process valued in Œ�1; 1�, and define the

sequence of stopping times

�k WD T ^ inf˚t � 0 W jY b

t j C jZbt j � k

�; k 2 N:

By Ito’s lemma and (5.12), it follows that

v.0; Y0; Z0/ D v��k; Y b

�k; Zb

�k

�Z �k

0

ŒbDyv C yDzv��t; Y b

t ; Zbt

dWt

�Z �k

0

.@t C Abt /v�t; Y b

t ; Zbt

dt

� v��k; Y b

�k; Zb

�k

�Z �k

0

ŒbDyv C yDzv��t; Y b

t ; Zbt

dWt :

Taking expected values and sending k to infinity, we get by Fatou’s lemma

v.0; Y0; Z0/ � lim infk!1 E

�v��k; Y b

�k; Zb

�k

�

� E�v�T; Y b

T ; ZbT

� D Ehe2Zb

T

i;

which proves the lemma. utWe are now able to prove the law of the iterated logarithm for double stochastic

integrals by a direct adaptation of the case of the Brownian motion. Set

h.t/ WD 2t log log1

tfor t > 0 :

Theorem 5.5. Let b be a predictable process valued in a bounded interval Œˇ0; ˇ1�

for some real parameters 0 � ˇ0 < ˇ1, and Xbt WD R t

0

R u0 bvdWvdWu. Then

ˇ0 � lim supt&0

2Xbt

h.t/� ˇ1 a.s.

Proof. We first show that the first inequality is an easy consequence of the secondone. Set N WD .ˇ0 C ˇ1/=2 � 0, and set ı WD .ˇ1 � ˇ0/=2. By the law of the iteratedlogarithm for the Brownian motion, we have

N D lim supt&0

2XN

t

h.t/� ı lim sup

t&0

2XQbt

h.t/C lim sup

t&0

2Xbt

h.t/;

5.2 Examples 61

where Qb WD ı�1. N�b/ is valued in Œ�1; 1�. It then follows from the second inequalitythat

lim supt&0

2Xbt

h.t/� N � ı D ˇ0:

We now prove the second inequality. Clearly, we can assume with no loss ofgenerality that kbk1 � 1. Let T > 0 and > 0 be such that 2T < 1. It followsfrom Doob’s maximal inequality for submartingales that for all ˛ � 0,

P

�max

0�t�T2Xb

t � ˛

�D P

�max

0�t�Texp.2Xb

t / � exp.˛/

�

� e�˛Ehe2Xb

T

i:

In view of Lemma 5.4, this provides

P

�max

0�t�T2Xb

t � ˛

�� e�˛E

he2X1

T

i

D e�.˛CT /.1 � 2T /� 12 : (5.13)

We have then reduced the problem to the case of the Brownian motion, and therest of this proof is identical to the first half of the proof of the law of the iteratedlogarithm for the Brownian motion. Take � , � 2 .0; 1/ and set for all k 2 N

˛k WD .1 C �/2h.�k/ and k WD Œ2�k.1 C �/��1:

Applying (5.13), we see that for all k 2 N,

P

�max

0�t��k2Xb

t � .1 C �/2h.�k/

�� e�1=2.1C�/

�1 C ��1

12 .�k log �/�.1C�/:

SinceP

k�0 k�.1C�/ < 1, it follows from the Borel-Cantelli lemma that, for almostall ! 2 ˝ , there exists a natural number K�;�.!/ such that for all k � K�;�.!/,

max0�t��k

2Xbt .!/ < .1 C �/2h.�k/:

In particular, for all t 2 .�kC1; �k�,

2Xbt .!/ < .1 C �/2h.�k/ � .1 C �/2 h.t/

�:


Hence,

lim supt&0

2Xbt

h.t/<

.1 C �/2

�a.s.,

and the required result follows by letting � tend to 1 and � to 0 along the rationals. ut

5.3 The Verification Argument for Optimal StoppingProblems

In this section, we develop the verification argument for finite horizon optimalstopping problems. Let T > 0 be a finite time horizon and Xt;x denote the solutionof the stochastic differential equation:

Xt;xs D x C

Z s

t

b.s; Xt;xs /ds C

Z s

t

�.s; Xt;xs /dWs; (5.14)

where b and � satisfy the usual Lipschitz and linear growth conditions. We denoteby P

Xt;xthe measure induced by this solution. Given the functions k; f W Œ0; T � �

Rd �! R and g W Rd �! R, we consider the optimal stopping problem

V.t; x/ WD sup�2T t

Œt;T �

E

�Z �

t

ˇ.t; s/f .s; Xt;xs /ds C ˇ.t; �/g.Xt;x

� /

�; (5.15)

whenever this expected value is well defined, where

ˇ.t; s/ WD e� R st k.r;X

t;xr /dr ; 0 � t � s � T:

By the results of the previous chapter, the corresponding dynamic programmingequation is

min f�@t v � Lv � f; v � gg D 0 on Œ0; T / � Rd ; v.T; :/ D g; (5.16)

where L is the second-order differential operator

Lv WD b � Dv C 1

2Tr��TD2v

� � kv:

Similar to Sect. 5.1, a function v will be called a supersolution (resp. subsolution)of (5.16) if

min f�@t v � Lv � f; v � gg � (resp. �) 0 and v.T; :/ � (resp. �) g:

5.3 The Verification Argument for Optimal Stopping Problems 63

Before stating the main result of this section, we observe that for many interestingexamples, it is known that the value function V does not satisfy the C 1;2 regularitywhich we have been using so far for the application of Ito’s formula. Therefore, inorder to state a result which can be applied to a wider class of problems, we shallenlarge in the following remark the set of function for which Ito’s formula still holdstrue.

Remark 5.6. Let v be a function in the Sobolev space W 1;2.S/. By definition, forsuch a function v, there is a sequence of functions .vn/n�1 � C 1;2.S/ such thatvn �! v uniformly on compact subsets of S, and

k@t vn � @t v

mkL2.S/ C kDvn � DvmkL2.S/ C kD2vn � D2vmkL2.S/ �! 0:

Then, Ito’s formula holds true for vn for all n � 1, and is inherited by v by sendingn ! 1.

Theorem 5.7. Let T < 1 and v 2 W 1;2.Œ0; T /;Rd /. Assume further that v and f

have quadratic growth. Then:

(i) If v is a supersolution of (5.16) dtxdPXt;x- a.e. for all .t; x/, then v � V .

(ii) If v is a solution of (5.16) dtxdPXt;x- a.e. for all .t; x/, then v D V , and

��t WD inf fs > t W v.s; Xs/ D g.Xs/g

is an optimal stopping time.

Proof. Let .t; x/ 2 Œ0; T / � Rd be fixed and denote ˇs WD ˇ.t; s/.

(i) For an arbitrary stopping time � 2 T tŒt;T / , we denote

�n WD � ^ inf˚s > t W jXt;x

s � xj > n�

:

By our regularity conditions on v, notice that Ito’s formula can be applied to itpiecewise. Then:

v.t; x/ D ˇ�n v.�n; Xt;x�n

/ �Z �n

t

ˇs.@t C L/v.s; Xt;xs /ds

�Z �n

t

ˇs.�TDv/.s; Xt;x

s /dWs

� ˇ�n v.�n; Xt;x�n

/ CZ �n

t

ˇsf .s; Xt;xs /ds

�Z �n

t

ˇs.�TDv/.s; Xt;x

s /dWs


by the supersolution property of v. Since .s; Xt;xs / is bounded on the stochastic

interval Œt; �n�, this provides

v.t; x/ � E

�ˇ�n v.�n; Xt;x

�n/ C

Z �n

t

ˇsf .s; Xt;xs /ds

�:

Notice that �n �! � a.s. Then, since f and v have quadratic growth, we maypass to the limit n ! 1 invoking the dominated convergence theorem, and weget:

v.t; x/ � E

�ˇT v.T; X

t;xT / C

Z T

t

ˇsf .s; Xt;xs /ds

�:

Since v.T; :/ � g by the supersolution property, this concludes the proof of (i).(ii) Let ��

t be the stopping time introduced in the theorem. Then, since v.T; :/ D g,it follows that ��

t 2 T tŒt;T �. Set

�nt WD ��

t ^ inf˚s > t W jXt;x

s � xj > n�

:

Observe that v > g on Œt; �nt / � Œt; ��

t /, and therefore �@t v � Lv � f D 0 onŒt; �n

t /. Then, proceeding as in the previous step, it follows from Ito’s formulathat:

v.t; x/ D E

"ˇ�n

tv.�n

t ; Xt;x

�nt

/ CZ �n

t

t

ˇsf .s; Xt;xs /ds

#:

Since �nt �! ��

t a.s. and f; v have quadratic growth, we may pass to the limitn ! 1 invoking the dominated convergence theorem. This leads to

v.t; x/ D E

�ˇT v.T; X

t;xT / C

Z T

t

ˇsf .s; Xt;xs /ds

�;

and the required result follows from the fact that v.T; :/ D g. ut

5.4 Examples of Optimal Stopping Problems with ExplicitSolutions

5.4.1 Perpetual American Options

The pricing problem of perpetual American put options reduces to the infinitehorizon optimal stopping problem:

P.t; s/ WD sup�2T t

Œt;1/

E�e�r.��t /.K � St;s

� /C�;

5.4 Examples of Optimal Stopping Problems with Explicit Solutions 65

where K > 0 is a given exercise price, St;s is defined by the Black-Scholes constantcoefficients model:

St;su WD s exp

r � �2

2

�.u � t/ C �.Wu � Wt /; u � t;

and r � 0, � > 0 are two given constants. By the time homogeneity of the problem,we see that

P.t; s/ D P.s/ WD sup�2TŒ0;1/

E�e�r� .K � S0;s

� /C�: (5.17)

In view this time independence, it follows that the dynamic programming corre-sponding to this problem is:

min

�v � .K � s/C; rv � rsDv � 1

2�2D2v

�D 0: (5.18)

In order to proceed to a verification argument, we now guess a solution to theprevious obstacle problem. From the nature of the problem, we search for a solutionof this obstacle problem defined by a parameter s0 2 .0; K/ such that

p.s/ D K � s for s 2 Œ0; s0� and rp � rsp0 � 1

2�2s2p00 D 0 on Œs0; 1/:

We are then reduced to solving a linear second-order ODE on Œs0; 1/, thusdetermining v by

p.s/ D As C Bs�2r=�2

for s 2 Œs0; 1/;

up to the two constants A and B . Notice that 0 � p � K . Then the constantA D 0 in our candidate solution because otherwise v �! 1 at infinity. Wefinally determine the constants B and s0 by requiring our candidate solution to becontinuous and differentiable at s�. This provides two equations:

Bs�2r=�2

0 D K � s0 and�2r=�2

Bs

�2r=�2�10 D �1;

which provide our final candidate

s0 D 2rK

2r C �2; p.s/ D .K � s/1Œ0;s0�.s/ C 1Œs0;1/.s/

�2s0

2r

s

s0

��2r

�2

: (5.19)

Notice that our candidate p is not twice differentiable at s0 as p00.s0�/ D 0 ¤p00.s0C/. However, by Remark 5.6, Ito’s formula still applies to p, and p satisfiesthe dynamic programming equation (5.18). We now show that


p D P with optimal stopping time �� WD inf˚t > 0 W p.S

0;st / D .K � S

0;st /C�:

(5.20)

Indeed, for an arbitrary stopping time � 2 TŒ0;1/, it follows from Ito’s formula that:

p.s/ D e�r�p.S0;s� / �

Z �

0

e�rt

�rp C rsp0 C 1

2�2s2p00

�.St /dt

�Z �

0

p0.St /�StdWt

� e�r� .K � St;s� /C �

Z �

0

p0.St /�St dWt

by the fact that p is a supersolution of the dynamic programming equation. Sincep0 is bounded, there is no need any localization to get rid of the stochastic integral,and we directly obtain by taking expected values that p.s/ � EŒe�r� .K � St;s

� /C�.By the arbitrariness of � 2 TŒ0;1/, this shows that p � P .

We next repeat the same argument with the stopping time ��, and we see thatp.s/ D EŒe�r��

.K � S0;s��

/C�, completing the proof of (5.20).

5.4.2 Finite Horizon American Options

Finite horizon optimal stopping problems rarely have an explicit solution. So, thefollowing example can be seen as a sanity check. In the context of the financialmarket of the previous subsection, we assume the instantaneous interest rate r D 0,and we consider an American option with payoff function g and maturity T > 0.Then the price of the corresponding American option is given by the optimalstopping problem:

P.t; s/ WD sup�2T t

Œt;T �

E�g.St;s

� /�: (5.21)

The corresponding dynamic programming equation is:

min

�v � g; �@t v � 1

2D2v

�D 0 on Œ0; T / � RC and v.T; :/ D g: (5.22)

Assuming further that g 2 W 1;2 and concave, we see that g is a solution of thedynamic programming equation. Then, provided that g satisfies suitable growthcondition, we see by a verification argument that P D p.

Notice that the previous result can be obtained directly by the Jensen inequalitytogether with the fact that S is a martingale.

Chapter 6Introduction to Viscosity Solutions

Throughout this chapter, we provide the main tools from the theory of viscositysolutions for the purpose of our applications to stochastic control problems. For adeeper presentation, we refer to the excellent overview paper by Crandall et al. [14].

6.1 Intuition Behind Viscosity Solutions

We consider a non-linear second order degenerate partial differential equation

.E/ F�x; u.x/; Du.x/; D2u.x/

� D 0 for x 2 O ;

where O is an open subset of Rd and F is a continuous map from O�R�Rd �Sd

�! R. We shall denote by d.O/ the closure of O in Rd . A crucial condition on F

is the so-called ellipticity condition:

Standing Assumption. For all .x; r; p/ 2 O � R � Rd and A; B 2 Sd ,

F.x; r; p; A/ � F.x; r; p; B/ whenever A � B:

The full importance of this condition will be made clear in Proposition 6.2 below.The first step towards the definition of a notion of weak solution to (E) is the

introduction of sub-and supersolutions.

Definition 6.1. A function u : O �! R is a classical supersolution (resp. subsolu-tion) of (E) if u 2 C 2.O/ and

F�x; u.x/; Du.x/; D2u.x/

� � (resp. �) 0 for x 2 O :

The theory of viscosity solutions is motivated by the following result, whosesimple proof is left to the reader.


67

68 6 Introduction to Viscosity Solutions

Proposition 6.2. Let u be a C 2.O/ function. Then the following claims areequivalent:

(i) u is a classical supersolution (resp. subsolution) of (E).(ii) For all pairs .x0; '/ 2 O�C 2.O/ such that x0 is a minimizer (resp. maximizer)

of the difference u � '/ on O, we have

F�x0; u.x0/; D'.x0/; D2'.x0/

� � (resp. �) 0:

6.2 Definition of Viscosity Solutions

For the convenience of the reader, we recall the definition of the semicontinuousenvelopes. For a locally bounded function u : O �! R, we denote by u� and u� thelower- and upper-semicontinuous envelopes of u. We recall that u� is the largestlower-semicontinuous minorant of u, u� is the smallest upper-semicontinuousmajorant of u, and

u�.x/ D lim infx0!x

u.x0/; u�.x/ D lim supx0!x

u.x0/ :

We are now ready for the definition of viscosity solutions. Observe that Claim (ii)in the above proposition does not involve the regularity of u. It therefore suggeststhe following weak notion of solution to (E).

Definition 6.3. Let u : O �! R be a locally bounded function:

(i) We say that u is a (discontinuous) viscosity supersolution of (E) if

F�x0; u�.x0/; D'.x0/; D2'.x0/

� � 0

for all pairs .x0; '/ 2 O � C 2.O/ such that x0 is a minimizer of the difference.u� � '/ on O.

(ii) We say that u is a (discontinuous) viscosity subsolution of (E) if

F�x0; u�.x0/; D'.x0/; D2'.x0/

� � 0

for all pairs .x0; '/ 2 O � C 2.O/ such that x0 is a maximizer of the difference.u� � '/ on O.

(iii) We say that u is a (discontinuous) viscosity solution of (E) if it is both aviscosity supersolution and subsolution of (E).

Notation. We will say that F�x; u�.x/; Du�.x/; D2u�.x/

� � 0 in the viscositysense whenever u� is a viscosity supersolution of (E). A similar notation will beused for subsolution.

6.3 First Properties 69

Remark 6.4. An immediate consequence of Proposition 6.2 is that any classicalsolution of (E) is also a viscosity solution of (E).

Remark 6.5. Clearly, the above definition is not changed if the minimum ormaximum are local and/or strict. Also, by a density argument, the test function canbe chosen in C 1.O/.

Remark 6.6. Consider the equation (EC): ju0.x/j � 1 D 0 on R. Then:

• The function f .x/ WD jxj is not a viscosity supersolution of (EC). Indeed, thetest function ' � 0 satisfies .f � '/.0/ D 0 � .f � '/.x/ for all x 2 R. Butj' 0.0/j D 0 6� 1.

• The function g.x/ WD �jxj is a viscosity solution of (EC). To see this, weconcentrate on the origin which is the only critical point. The supersolutionproperty is obviously satisfied as there is no smooth function which satisfies theminimum condition. As for the subsolution property, we observe that whenever' 2 C 1.R/ satisfies .g�'/.0/ D max.g�'/, then j' 0.0/j � 1, which is exactlythe viscosity subsolution property of g.

• Similarly, the function f is a viscosity solution of the equation (E�): �ju0.x/j C1 D 0 on R.

In Sect. 7.1, we will show that the value function V is a viscosity solution of theDPE (3.21) under the conditions of Theorem 3.6 (except the smoothness assumptionon V ). We also want to emphasize that proving that the value function is a viscositysolution is almost as easy as proving that it is a classical solution when V is knownto be smooth.

6.3 First Properties

We now turn to two important properties of viscosity solutions: the change ofvariable formula and the stability result.

Proposition 6.7. Let u be a locally bounded (discontinuous) viscosity supersolutionof (E). If f is a C 1.R/ function with Df ¤ 0 on R, then the function v WD f �1 ı uis a (discontinuous)

– Viscosity supersolution, when Df > 0

– Viscosity subsolution, when Df < 0

of the equation

K.x; v.x/; Dv.x/; D2v.x// D 0 for x 2 O;

where

K.x; r; p; A/ WD F�x; f .r/; Df .r/p; D2f .r/pp0 C Df .r/A

�:


We leave the easy proof of this proposition to the reader. The next result showshow limit operations with viscosity solutions can be performed very easily.

Theorem 6.8. Let u" be a lower-semicontinuous viscosity supersolution of theequation

F"

�x; u".x/; Du".x/; D2u".x/

� D 0 for x 2 O;

where .F"/">0 is a sequence of continuous functions satisfying the ellipticitycondition. Suppose that ."; x/ 7�! u".x/ and ."; z/ 7�! F".z/ are locally bounded,and define

u.x/ WD lim inf.";x0/!.0;x/

u".x0/ and F .z/ WD lim sup

.";z0/!.0;z/F".z

0/:

Then, u is a lower-semicontinuous viscosity supersolution of the equation

F�x; u.x/; Du.x/; D2u.x/

� D 0 for x 2 O:

A similar statement holds for subsolutions.

Proof. The fact that u is a lower-semicontinuous function is left as an exercise forthe reader. Let ' 2 C 2.O/ and Nx be a strict minimizer of the difference u � '.By definition of u, there is a sequence ."n; xn/ 2 .0; 1� � O such that

."n; xn/ �! .0; Nx/ and u"n.xn/ �! u. Nx/:

Consider some r > 0 together with the closed ball NB with radius r , centered at Nx.Of course, we may choose jxn � Nxj < r for all n � 0. Let Nxn be a minimizer ofu"n � ' on NB . We claim that

Nxn �! Nx and u"n. Nxn/ �! u. Nx/ as n ! 1: (6.1)

Before verifying this, let us complete the proof. We first deduce that Nxn is an interiorpoint of NB for large n, so that Nxn is a local minimizer of the difference u"n � '. Then

F"n

� Nxn; u"n. Nxn/; D'. Nxn/; D2'. Nxn/� � 0;

and the required result follows by taking limits and using the definition of F .It remains to prove claim (6.1). Recall that .xn/n is valued in the compact set NB .

Then, there is a subsequence, still named .xn/n, which converges to some Qx 2 NB .We now prove that Qx D Nx and obtain the second claim in (6.1) as a by-product.Using the fact that Nxn is a minimizer of u"n � ' on NB , together with the definition ofu, we see that

6.3 First Properties 71

0 D .u � '/. Nx/ D limn!1 .u"n � '/ .xn/

� lim supn!1

.u"n � '/ . Nxn/

� lim infn!1 .u"n � '/ . Nxn/

� .u � '/. Qx/ :

We now obtain (6.1) from the fact that Nx is a strict minimizer of the difference.u � '/. ut

Observe that the passage to the limit in partial differential equations written inthe classical or the generalized sense usually requires much more technicalities, asone has to ensure convergence of all the partial derivatives involved in the equation.The above stability result provides a general method to pass to the limit when theequation is written in the viscosity sense, and its proof turns out to be remarkablysimple.

A possible application of the stability result is to establish the convergence ofnumerical schemes. In view of the simplicity of the above statement, the notion ofviscosity solutions provides a nice framework for such questions. This issue will bestudied later in Chap. 12.

The main difficulty in the theory of viscosity solutions is the interpretation ofthe equation in the viscosity sense. First, by weakening the notion of solution tothe second-order nonlinear PDE (E), we are enlarging the set of solutions, and onehas to guarantee that uniqueness still holds (in some convenient class of functions).This issue will be discussed in the subsequent Sect. 6.4. We conclude this sectionby the following result whose proof is trivial in the classical case, but needs sometechnicalities when stated in the viscosity sense.

Proposition 6.9. Let A � Rd1 and B � R

d2 be two open subsets, and let u : A � B

�! R be a lower semicontinuous viscosity supersolution of the equation:

F�x; y; u.x; y/; Dy u.x; y/; D2

yu.x; y/�

� 0 on A � B;

where F is a continuous elliptic operator. Then, for all fixed x0 2 A, the functionv.y/ WD u.x0; y/ is a viscosity supersolution of the equation:

F�x0; y; v.y/; Dv.y/; D2v.y/

� � 0 on B:

A similar statement holds for the subsolution property.

Proof. Fix x0 2 A, set v.y/ WD u.x0; y/, and let y0 2 B and f 2 C 2.B/ be suchthat

.v � f /.y0/ < .v � f /.y/ for all y 2 J n fy0g; (6.2)


where J is an arbitrary compact subset of B containing y0 in its interior. For eachinteger n, define

'n.x; y/ WD f .y/ � njx � x0j2 for .x; y/ 2 A � B;

and let .xn; yn/ be defined by

.u � 'n/.xn; yn/ D minI�J

.u � 'n/;

where I is a compact subset of A containing x0 in its interior. We claim that

.xn; yn/ �! .x0; y0/ and u.xn; yn/ �! u.x0; y0/ as n �! 1: (6.3)

Before proving this, let us complete the proof. Since .x0; y0/ is an interior point ofA � B , it follows from the viscosity property of u that

0 � F�xn; yn; u.xn; yn/; Dy'n.xn; yn/; D2

y'n.xn; yn/�

D F�xn; yn; u.xn; yn/; Df .yn/; D2f .yn/

�;

and the required result follows by sending n to infinity.We now turn to the proof of (6.3). Since the sequence .xn; yn/n is valued in the

compact subset A � B , we have .xn; yn/ �! . Nx; Ny/ 2 A � B , after passing to asubsequence. Observe that

u.xn; yn/ � f .yn/ � u.xn; yn/ � f .yn/ C njxn � x0j2D .u � 'n/.xn; yn/

� .u � 'n/.x0; y0/ D u.x0; y0/ � f .y0/ :

Taking the limits, it follows from the lower semicontinuity of u that

u. Nx; Ny/ � f . Ny/ � lim infn!1 u.xn; yn/ � f .yn/ C njxn � x0j2

� lim supn!1

u.xn; yn/ � f .yn/ C njxn � x0j2

� u.x0; y0/ � f .y0/: (6.4)

Since u is lower semicontinuous, this implies that u. Nx; Ny/�f . Ny/Clim infn!1 njxn�x0j2 � u.x0; y0/ � f .y0/. Then, we must have Nx D x0, and

.v � f /. Ny/ D u.x0; Ny/ � f . Ny/ � .v � f /.y0/;

which implies that Ny D y0 in view of (6.2), and njxn � x0j2 �! 0. We also deducefrom inequalities (6.4) that u.xn; yn/ �! u.x0; y0/, concluding the proof of (6.3). ut

6.4 Comparison Result and Uniqueness 73

6.4 Comparison Result and Uniqueness

In this section, we show that the notion of viscosity solutions is consistent withthe maximum principle for a wide class of equations. Once we will have such aresult, the reader must be convinced that the notion of viscosity solutions is a goodweakening of the notion of classical solution.

We recall that the maximum principle is a stronger statement than uniqueness,i.e., any equation satisfying a comparison result has no more than one solution.

In the viscosity solutions literature, the maximum principle is rather calledcomparison principle.

6.4.1 Comparison of Classical Solutions in a Bounded Domain

Let us first review the maximum principle in the simplest classical sense.

Proposition 6.10. Assume that O is an open bounded subset of Rd , and the non-linearity F.x; r; p; A/ is elliptic and strictly increasing in r . Let u; v 2 C 2

�cl.O/

�

be classical subsolution and supersolution of (E), respectively, with u � v on @O.Then u � v on cl.O/.

Proof. Our objective is to prove that

M WD supcl.O/

.u � v/ � 0:

Assume to the contrary that M > 0. Then since cl.O/ is a compact subset of Rd ,and u � v � 0 on @O, we have

M D .u � v/.x0/ for some x0 2 O with D.u � v/.x0/ D 0; D2.u � v/.x0/ � 0:

(6.5)

Then, it follows from the viscosity properties of u and v that:

F�x0; u.x0/; Du.x0/; D2u.x0/

� � 0 � F�x0; v.x0/; Dv.x0/; D2v.x0/

�

� F�x0; u.x0/ � M; Du.x0/; D2u.x0/

�;

where the last inequality follows crucially from the ellipticity of F . This providesthe desired contradiction, under our condition that F is strictly increasing in r . ut

The objective of this section is to mimic the previous proof in the sense ofviscosity solutions.


6.4.2 Semijets Definition of Viscosity Solutions

We first need to develop a convenient alternative definition of viscosity solutions.For x0 2 O, r 2 R, p 2 R

d , and A 2 Sd , we introduce the quadratic function:

q.y; r; p; A/ WD r C p � y C 1

2Ay � y; y 2 R

d :

For v 2 LSC.O/, let .x0; '/ 2 O � C 2.O/ be such that x0 is a local minimizerof the difference .v � '/ in O. Then, defining p WD D'.x0/ and A WD D2'.x0/, itfollows from a second-order Taylor expansion that:

v.x/ � q�x � x0; v.x0/; p; A

�C ı�jx � x0j2�:

Motivated by this observation, we introduce the subjet J �O v.x0/ by

J �O v.x0/ WD

n.p; A/ 2 R

d � Sd W v.x/ � q�x � x0; v.x0/; p; A

�C ı�jx � x0j2�o:

(6.6)

Similarly, we define the superjet J CO u.x0/ of a function u 2 USC.O/ at the point

x0 2 O by

J CO u.x0/ WD

n.p; A/ 2 R

d � Sd W u.x/ � q�x � x0; u.x0/; p; A

�C ı�jx � x0j2�o:

(6.7)

Then, it can prove that a function v 2 LSC.O/ is a viscosity supersolution of theequation (E) if and only if

F.x; v.x/; p; A/ � 0 for all .p; A/ 2 J �Ov.x/:

The nontrivial implication of the previous statement requires to construct, for every.p; A/ 2 J �

O v.x0/, a smooth test function ' such that the difference .v � '/ has alocal minimum at x0. We refer to Fleming and Soner [20], Lemma V.4.1 p. 211.

A symmetric statement holds for viscosity subsolutions. By continuity consider-ations, we can even enlarge the semijets JO w.x0/ to the following closure:

NJO w.x/ WDn.p; A/ 2 R

d � Sd W .xn; w.xn/; pn; An/ �! .x; w.x/; p; A/

for some sequence .xn; pn; An/n � Graph.JO w/o;

where .xn; pn; An/ 2 Graph.JO w/ means that .pn; An/ 2 JO w.xn/. The followingresult is obvious, and provides an equivalent definition of viscosity solutions.


Proposition 6.11. Consider an elliptic nonlinearity F , and let u 2 USC.O/, v 2LSC.O/:

(i) Assume that F is lower semicontinuous. Then, u is a viscosity subsolution of(E) if and only if

F.x; u.x/; p; A/ � 0 for all x 2 O and .p; A/ 2 NJ CO u.x/:

(ii) Assume that F is upper semicontinuous. Then, v is a viscosity supersolution of(E) if and only if

F.x; v.x/; p; A/ � 0 for all x 2 O and .p; A/ 2 NJ �Ov.x/:

6.4.3 The Crandall–Ishii’s Lemma

The major difficulty in mimicking the proof of Proposition 6.10 is to derive ananalogous statement to (6.5) without involving the smoothness of u and v, as thesefunctions are only known to be upper- and lowern semicontinuous in the context ofviscosity solutions.

This is provided by the following result due to M. Crandall and I. Ishii. For asymmetric matrix, we denote by jAj WD supf.A�/ � � W j�j � 1g.

Lemma 6.12. Let O be an open subset of Rd . Given u 2 USC.O/ and v 2

LSC.O/, we assume for some .x0; y0/ 2 O2, ' 2 C 2�cl.O/2

�that:

.u � v � '/.x0; y0/ D maxO2

.u � v � '/: (6.8)

Then, for each " > 0, there exist A; B 2 Sd such that

.Dx'.x0; y0/; A/ 2 NJ CO u.x0/; .�Dy'.x0; y0/; B/ 2 NJ �

O v.y0/;

and the following inequality holds in the sense of symmetric matrices in S2d :

� �"�1 C ˇˇD2'.x0; y0/

ˇˇ� I2d �

�A 0

0 �B

�� D2'.x0; y0/ C "D2'.x0; y0/

2:

Proof. See Sect. 6.7. utWe will be applying Lemma 6.12 in the particular case

'.x; y/ WD ˛

2jx � yj2 for x; y 2 O: (6.9)


Intuitively, sending ˛ to 1, we expect that the maximization of .u.x/ � v.y/ �'.x; y/ on O2 reduces to the maximization of .u � v/ on O as in (6.5). Then, taking"�1 D ˛, we directly compute that the conclusions of Lemma 6.12 reduce to

.˛.x0 � y0/; A/ 2 NJ CO u.x0/; .˛.x0 � y0/; B/ 2 NJ �

Ov.y0/; (6.10)

and

� 3˛

�Id 0

0 Id

��

A 0

0 �B

�� 3˛

�Id �Id

�Id Id

�: (6.11)

Remark 6.13. If u and v were C 2 functions in Lemma 6.12, the first-and-secondorder condition for the maximization problem (6.8) with the test function (6.9) isDu.x0/ D ˛.x0 � y0/, Dv.x0/ D ˛.x0 � y0/, and

�D2u.x0/ 0

0 �D2v.y0/

�� ˛

�Id �Id

�Id Id

�:

Hence, the right-hand side inequality in (6.11) is worsening the previous second-order condition by replacing the coefficient ˛ by 3˛.

Remark 6.14. The right-hand side inequality of (6.11) implies that

A � B: (6.12)

To see this, take an arbitrary � 2 Rd , and denote by �T its transpose. From right-

hand side inequality of (6.11), it follows that

0 � .�T; �T/

�A 0

0 �B

��

�

�D .A�/ � � � .B�/ � �:

6.4.4 Comparison of Viscosity Solutions in a Bounded Domain

We now prove a comparison result for viscosity sub- and supersolutions by usingLemma 6.12 to mimic the proof of Proposition 6.10. The statement will be provedunder the following conditions on the nonlinearity F which will be used at Step 3of the subsequent proof.

Assumption 6.15. (i) There exists � > 0 such that

F.x; r; p; A/�F.x; r 0; p; A/ � �.r�r 0/ for all r � r 0; .x; p; A/ 2 O�Rd �Sd :

(ii) There is a function $ W RC �! RC with $.0C/ D 0, such that


F.y; r; ˛.x � y/; B/ � F.x; r; ˛.x � y/; A/ � $�˛jx � yj2 C jx � yj�

for all x; y 2 O; r 2 R ,and A; B satisfying (6.11):

Remark 6.16. Assumption 6.15(ii) implies that the nonlinearity F is elliptic. To seethis, notice that for A � B , �; � 2 R

d , and " > 0, we have

A� � � � .B C "Id /� � � � B� � � � .B C "Id /� � �

D 2� � B.� � �/ C B.� � �/ � .� � �/ � "j�j2� "�1jB.� � �/j2 C B.� � �/ � .� � �/

� jBj �1 C "�1jBj� j� � �j2:

For 3˛ � .1 C "�1jBj/jBj, the latter inequality implies the right-hand side of (6.11)holds true with .A; B C "Id /. For " sufficiently small, the left-hand side of (6.11) isalso true with .A; B C "Id / if in addition ˛ > jAj _ jBj. Then

F.x � ˛�1p; r; p; B C "I / � F.x; r; p; A/ � $�˛�1.jpj2 C jpj/� ;

which provides the ellipticity of F by sending ˛ ! 1 and " ! 0.

Theorem 6.17. Let O be an open bounded subset of Rd and let F be an ellipticoperator satisfying Assumption 6.15. Let u 2 USC.O/ and v 2 LSC.O/ be viscositysubsolution and supersolution of the equation (E), respectively. Then

u � v on @O H) u � v on NO WD cl.O/:

Proof. As in the proof of Proposition 6.10, we assume to the contrary that

ı WD .u � v/.z/ > 0 for some z 2 O: (6.13)

Step 1. For every ˛ > 0, it follows from the upper semicontinuity of the difference.u � v/ and the compactness of NO that

M˛ WD supO�O

nu.x/ � v.y/ � ˛

2jx � yj2

o

D u.x˛/ � v.y˛/ � ˛

2jx˛ � y˛j2 (6.14)

for some .x˛; y˛/ 2 NO� NO. Since NO is compact, there is a subsequence .xn; yn/ WD.x˛n ; y˛n /, n � 1, which converges to some . Ox; Oy/ 2 NO � NO. We shall prove in Step4 below that

Ox D Oy; ˛njxn � ynj2 �! 0; and M˛n �! .u � v/. Ox/ D supO

.u � v/: (6.15)


Then, since u � v on @O and

ı � M˛n D u.xn/ � v.yn/ � ˛n

2jxn � ynj2 (6.16)

by (6.13), it follows from the first claim in (6.15) that .xn; yn/ 2 O � O.

Step 2. Since the maximizer .xn; yn/ of M˛n defined in (6.14) is an interior pointto O � O, it follows from Lemma 6.12 that there exist two symmetric matricesAn; Bn 2 Sn satisfying (6.11) such that .xn; ˛n.xn � yn/; An/ 2 NJ C

O u.xn/ and.yn; ˛n.xn � yn/; Bn/ 2 NJ �

Ov.yn/. Then, since u and v are viscosity subsolutionand supersolution, respectively, it follows from the alternative definition of viscositysolutions in Proposition 6.11 that

F .xn; u.xn/; ˛n.xn � yn/; An/ � 0 � F .yn; v.yn/; ˛n.xn � yn/; Bn/ : (6.17)

Step 3. We first use the strict monotonicity Assumption 6.15(i) to obtain

�ı � ��u.xn/ � v.yn/

� � F .xn; u.xn/; ˛n.xn � yn/; An/

�F .xn; v.yn/; ˛n.xn � yn/; An/ :

By (6.17), this provides

�ı � F .yn; v.yn/; ˛n.xn � yn/; Bn/ � F .xn; v.yn/; ˛n.xn � yn/; An/ :

Finally, in view of Assumption 6.15(ii) this implies that

�ı � $�˛njxn � ynj2 C jxn � ynj� :

Sending n to infinity, this leads to the desired contradiction of (6.13) and (6.15).

Step 4. It remains to prove the claims (6.15). By the upper semicontinuity of thedifference .u � v/ and the compactness of NO, there exists a maximizer x� of thedifference .u � v/. Then

.u � v/.x�/ � M˛n D u.xn/ � v.yn/ � ˛n

2jxn � ynj2:

Sending n ! 1, this provides

N WD 1

2lim sup

n!1˛njxn � ynj2 � lim sup

n!1u.x˛n/ � v.y˛n/ � .u � v/.x�/

� u. Ox/ � v. Oy/ � .u � v/.x�/I

in particular, N < 1 and Ox D Oy. Using the definition of x� as a maximizer of .u�v/,we see that:


0 � N � .u � v/. Ox/ � .u � v/.x�/ � 0:

Then Ox is a maximizer of the difference .u � v/ and M˛n �! supO.u � v/. utWe list below two interesting examples of operators F which satisfy the

conditions of the above theorem:

Example 6.18. Assumption 6.15 is satisfied by the nonlinearity

F.x; r; p; A/ D �r C H.p/

for any continuous function H W Rd �! R, and � > 0.In this example, the condition � > 0 is not needed when H is a convex and

H.D'.x// � ˛ < 0 for some ' 2 C 1.O/. This result can be found in [2].

Example 6.19. Assumption 6.15 is satisfied by

F.x; r; p; A/ D �Tr�� 0.x/A

�C �r;

where � W Rd �! Sd is a Lipschitz function, and � > 0. Condition (i) of

Assumption 6.15 is obvious. To see that condition (ii) is satisfied, we consider.A; B; ˛/ 2 Sd � Sd � R

�C satisfying (6.11). We claim that

TrhMM TA � NN TB

i� 3˛jM � N j2 D 3˛

dX

i;j D1

.M � N /2ij :

To see this, observe that the matrix

C WD

NN T NM T

MN T MM T

!

is a non-negative matrix in Sd . From the right hand side inequality of (6.11), thisimplies that

TrhMM TA � NN TB

iD Tr

�C

�A 0

0 �B

�

� 3˛Tr

�C

�Id �Id

�Id Id

�

D 3˛Trh.M � N /.M � N /T

iD 3˛jM � N j2:


6.5 Comparison in Unbounded Domains

When the domain O is unbounded, a growth condition on the functions u and vis needed. Then, by using the growth at infinity, we can build on the proof ofTheorem 6.17 to obtain a comparison principle. The following result shows howto handle this question in the case of a sub-quadratic growth. We emphasize that thepresent argument can be adapted to alternative growth conditions.

The following condition differs from Assumption 6.15 only in its part (ii)where the constant 3 in (6.11) is replaced by 4 in (6.18). Thus, the followingAssumption 6.20(ii) is slightly stronger than Assumption 6.15(ii).

Assumption 6.20. (i) There exists � > 0 such that

F.x; r; p; A/�F.x; r 0; p; A/ � �.r�r 0/ for all r � r 0; .x; p; A/ 2 O�Rd �Sd :

(ii) There is a function $ W RC �! RC with $.0C/ D 0, such that

F.y; r; ˛.x � y/; B/ � F.x; r; ˛.x � y/; A/ � $�˛jx � yj2 C jx � yj�

for all x; y 2 O; r 2 R and A; B satisfying

�4˛

�Id 0

0 Id

��

A 0

0 �B

�� 4˛

�Id �Id

�Id Id

�: (6.18)

Theorem 6.21. Let F be a uniformly continuous elliptic operator satisfyingAssumption 6.20. Let u 2 USC.O/ and v 2 LSC.O/ be viscosity subsolutionand supersolution of the equation (E), respectively, with ju.x/j C jv.x/j D ı.jxj2/as jxj ! 1. Then

u � v on @O H) u � v on cl.O/:

Proof. We assume to the contrary that

ı WD .u � v/.z/ > 0 for some z 2 Rd ; (6.19)

and we work towards a contradiction. Let

M˛ WD supx;y2Rd

u.x/ � v.y/ � �.x; y/;

where

�.x; y/ WD 1

2

�˛jx � yj2 C "jxj2 C "jyj2� :

6.5 Comparison in Unbounded Domains 81

1. Since u.x/ D ı.jxj2/ and v.y/ D ı.jyj2/ at infinity, there is a maximizer .x˛; y˛/

for the previous problem:

M˛ D u.x˛/ � v.y˛/ � �.x˛; y˛/:

Moreover, there is a sequence ˛n ! 1 such that

.xn; yn/ WD .x˛n ; y˛n/ �! . Ox; Oy/;

and, similar to Step 4 of the proof of Theorem 6.17, we can prove that Ox D Oy,

˛njxn � ynj2 �! 0; and M˛n �! M1 WD supx2Rd

.u � v/.x/ � "jxj2: (6.20)

Notice that

lim supn!1

M˛n D lim supn!1

fu.xn/ � v.yn/ � �.xn; yn/g

� lim supn!1

fu.xn/ � v.yn//g

� lim supn!1

u.xn/ � lim infn!1 v.yn/

� .u � v/. Ox/:

Since u � v on @O, and

M˛n � ı � "jzj2 > 0;

by (6.19), we deduce that Ox 62 @O and therefore .xn; yn/ is a local maximizer ofu � v � �.

2. By the Crandall–Ishii Lemma 6.12, there exist An; Bn 2 Sn, such that

.Dx�.xn; yn/; An/ 2 NJ 2;CO u.tn; xn/;

��Dy�.xn; yn/; Bn

� 2 NJ 2;�O v.tn; yn/; (6.21)

and

� .˛ C jD2�.x0; y0/j/I2d ��

An 0

0 �Bn

�

� D2�.xn; yn/ C 1

˛D2�.xn; yn/2: (6.22)

In the present situation, we immediately calculate that

Dx�.xn; yn/ D ˛.xn � yn/ C "xn; ; � Dy�.xn; yn/ D ˛.xn � yn/ � "yn


and

D2�.xn; yn/ D ˛

�Id �Id

�Id Id

�C " I2d ;

which reduces the right hand side of (6.22) to

�An 0

0 �Bn

�� .3˛ C 2"/

�Id �Id

�Id Id

�C�

" C "2

˛

�I2d ; (6.23)

while the left land side of (6.22) implies

� 3˛I2d ��

An 0

0 �Bn

�: (6.24)

3. By (6.21) and the viscosity properties of u and v, we have

F.xn; u.xn/; ˛n.xn � yn/ C "xn; An/ � 0;

F.yn; v.yn/; ˛n.xn � yn/ � "yn; Bn/ � 0:

Using Assumption 6.20(i) together with the uniform continuity of H , this impliesthat

��u.xn/ � v.xn/

� � F�yn; u.xn/; ˛n.xn � yn/; QBn

�

�F�xn; u.xn/; ˛n.xn � yn/; QAn

�C c."/;

where c.:/ is a modulus of continuity of F , and QAn WD An � 2"In, QBn WD Bn C2"In. By (6.23) and (6.24), we have

�4˛I2d �� QAn 0

0 � QBn

�� 4˛

�Id �Id

�Id Id

�;

for small ". Then, it follows from Assumption 6.20(ii) that

��u.xn/ � v.xn/

� � $�˛njxn � ynj2 C jxn � ynj�C c."/:

By sending n to infinity, it follows from (6.20) that:

c."/ � ��M1 C j Oxj2� � �M1 � �

�u.z/ � v.z/ � "jzj2�;

and we get a contradiction of (6.19) by sending " to zero. ut

6.6 Useful Applications 83

6.6 Useful Applications

We conclude this section by two consequences of the above comparison results,which are trivial properties in the context of classical solutions.

Lemma 6.22. Let O be an open interval of R, and U : O �! R be a lower-semicontinuous viscosity supersolution of the equation DU � 0 on O. Then U isnondecreasing on O.

Proof. For each " > 0, define W.x/ WD U.x/ C "x, x 2 O. Then W satisfies in theviscosity sense DW � " in O, i.e., for all .x0; '/ 2 O � C 1.O/ such that

.W � '/.x0/ D minx2O.W � '/.x/; (6.25)

we have D'.x0/ � ". This proves that ' is strictly increasing in a neighborhood Vof x0. Let .x1; x2/ � V be an open interval containing x0. We intend to prove that

W.x1/ < W.x2/; (6.26)

which provides the required result from the arbitrariness of x0 2 O.To prove (6.26), suppose to the contrary that W.x1/ � W.x2/, and the consider

the function v.x/ D W.x2/ which solves the equation

Dv D 0 on the open interval .x1; x2/:

Together with the boundary conditions v.x1/ D v.x2/ D W.x2/. Observe that W

is a lower-semicontinuous viscosity supersolution of the above equation. From thecomparison result of Example 6.18, this implies that

supŒx1; x2�

.v � W / D max f.v � W /.x1/; .v � W /.x2/g � 0:

Hence W.x/ � v.x/ D W.x2/ for all x 2 Œx1; x2�. Applying this inequality atx0 2 .x1; x2/ and recalling that the test function ' is strictly increasing on Œx1; x2�,we get

.W � '/.x0/ > .W � '/.x2/;

contradicting (6.25). utLemma 6.23. Let O be an open interval of R, and U : O �! R be a lower-semicontinuous viscosity supersolution of the equation �D2U � 0 on O. Then U

is concave on O.

Proof. Let a < b be two arbitrary elements in O, and consider some " > 0 togetherwith the function


v.s/ WDU.a/

�e

p

".b�s/ � e�

p

".b�s/�

C U.b/�

ep

".s�a/ � e�

p

".s�a/�

ep

".b�a/ � e�

p

".b�a/for a � s � b:

Clearly, v solves the equation

"v � D2v D 0 on .a; b/; v D U on fa; bg:

Since U is lower semicontinuous, it is bounded from below on the interval Œa; b�.Therefore, by possibly adding a constant to U , we can assume that U � 0, so thatU is a lower-semicontinuous viscosity supersolution of the above equation. It thenfollows from the comparison Theorem 7.6 that:

supŒa;b�

.v � U / D max f.v � U /.a/; .v � U /.b/g D 0:

Hence,

U.s/ � v.s/ DU.a/

�e

p".b�s/ � e�p

".b�s/�

C U.b/�

ep

".s�a/ � e�p".s�a/

�

ep

".b�a/ � e�p".b�a/

and by sending " to zero, we see that

U.s/ � .U.b/ � U.a//s � a

b � aC U.a/

for all s 2 Œa; b�. Let � be an arbitrary element of the interval [0,1], and set s WD�a C .1 � �/b. The last inequality takes the form

U��a C .1 � �/b

� � �U.a/ C .1 � �/U.b/;

proving the concavity of U . ut

6.7 Proof of the Crandall–Ishii’s Lemma

We start with two lemmas. We say that a function f is ��semiconvex if x 7�!f .x/ C .�=2/jxj2 is convex.

Lemma 6.24. Let f W RN �! R be a ��semiconvex function, for some � 2 R,and assume that f .x/ � 1

2Bx � x � f .0/ for all x 2 R

N . Then there exists X 2 SN

such that

.0; X/ 2 J2;C

f .0/ \ J2;�

f .0/ and ��IN � X � B:

6.7 Proof of the Crandall–Ishii’s Lemma 85

Our second lemma requires to introduce the following notion. For a functionv W RN �! R and � > 0, the corresponding ��sup-convolution is defined by:

Ov�.x/ WD supy2RN

v.y/ � �

2jx � yj2

�:

Observe that

Ov�.x/ C �

2jxj2 D sup

y2RN

v.y/ � �

2jyj2 C �x � y

�

is convex, as the supremum of linear functions. Then

Ov� is � � semiconvex: (6.27)

In [14], the following property is referred to as the magical property of the sup-convolution.

Lemma 6.25. Let � > 0, v be a bounded lower-semicontinuous function and Ov� thecorresponding ��sup-convolution.

(i) If .p; X/ 2 J 2;C Ov.x/ for some x 2 RN , then

.p; X/ 2 J 2;Cv�x C p

�

�and Ov�.x/ D v.x C p=�/ � 1

2�jpj2:

(ii) For all x 2 RN , we have .0; X/ 2 NJ 2;C Ov.x/implies that .0; X/ 2 NJ 2;Cv.x/.

Before proving the above lemmas, we show how they imply the Crandall–Ishii’slemma that we reformulate in a more symmetric way.

Lemma 6.12. Let O be an open locally compact subset of Rd and u1; u2 2

USC.O/. We denote w.x1; x2/ WD u1.x1/ C u2.x2/ and we assume for some' 2 C 2

�cl.O/2

�, and x0 D .x0

1 ; x02/ 2 O � O that

.w � '/.x0/ D maxO2

.w � '/:

Then, for each " > 0, there exist X1; X2 2 Sd such that

.Dxi '.x0/; Xi / 2 NJ 2;CO ui .x

0i /; i D 1; 2;

and

� �"�1 C ˇˇD2'.x0/

ˇˇ� I2d �

�X1 0

0 X2

�� D2'.x0/ C "D2'.x0/2:


Proof.

Step 1. We first observe that we may reduce the problem to the case

O D Rd ; x0 D 0; u1.0/ D u2.0/ D 0; and '.x/ D 1

2Ax � x for some A 2 Sd :

The reduction to x0 D 0 follows from an immediate change of coordinates. Chooseany compact subset of K � O containing the origin and set Nui D ui on K and �1otherwise, i D 1; 2. Then, the problem can be stated equivalently in terms of thefunctions Nui which are now defined on R

d and take values on the extended real line.Also by defining

NNui .xi / WD Nui .xi / � ui .0/ � Dxi '.0/ and N'.x/ WD '.x/ � '.0/ � D'.0/ � x;

we may reformulate the problem equivalently with NNui .xi / D 0 and N'.x/ D12D2'.0/x �x Cı.jxj2/. Finally, defining NN'.x/ WD Ax �x with A WD D2'.0/C�I2d

for some � > 0, it follows that

NNu1.x1/ C NNu2.x2/ � NN'.x1; x2/ < NNu1.x1/ C NNu2.x2/ � N'.x1; x2/

� NNu1.0/ C NNu2.0/ � N'.0/ D 0:

Step 2. From the reduction of the previous step, we have

2w.x/ � Ax � x

D A.x � y/ � .x � y/Ay � y � 2Ay � .y � x/

� A.x � y/ � .x � y/Ay � y C "A2y � y C 1

"jx � yj2

D A.x � y/ � .x � y/ C 1

"jx � yj2 C .A C "A2/y � y

� ."�1 C jAj/jx � yj2 C .A C "A2/y � y:

Set � WD "�1 C jAj and B WD A C "A2. The latter inequality implies the followingproperty of the sup-convolution:

Ow�.y/ � 1

2By � y � Ow.0/ D 0:

Step 3. Recall from (6.27) that Ow� is ��semiconvex. Then, it follows from Lemma

6.24 that there exist X 2 S2d such that .0; X/ 2 J2;C Ow�.0/ \ J

2;� Ow�.0/ and��I2d � X � B . Moreover, it is immediately checked that Ow�.x1; x2/ D Ou�

1.x1/ COu�

2.x2/, implying that X is block diagonal with blocs X1; X2 2 Sd . Hence,

6.7 Proof of the Crandall–Ishii’s Lemma 87

�."�1 C jAj/I2d ��

X1 0

0 X2

�� A C "A2;

and .0; Xi/ 2 J2;C Ou�

i .0/ for i D 1; 2 which, by Lemma 6.25, implies that .0; Xi/ 2J

2;Cu�

i .0/. utWe continue by turning to the proofs of Lemmas 6.24 and 6.25. The main tools

which will be used are the following properties of any semiconvex function ' WR

N �! R whose proofs are reported in [14]:

• Aleksandrov’s lemma. ' is twice differentiable a.e.• Jensen’s lemma. if x0 is a strict maximizer of ', then for every r; ı > 0, the set

˚ Nx 2 B.x0; r/ W x 7�! '.x/ C p � x has a local maximum at Nx for some p 2 Bı

�

has positive measure in RN .

Proof of Lemma 6.24. Notice that '.x/ WD f .x/ � 12Bx � x � jxj4 has a strict

maximum at x D 0. Localizing around the origin, we see that ' is a semiconvexfunction. Then, for every ı > 0, by the above Aleksandrov’s and Jensen’s lemmas,there exists qı and xı such that

qı; xı 2 Bı; D2'.xı/ exists, and '.xı/ C qı � xı D loc-maxf'.x/ C qı � xg:

We may then write the first- and second-order optimality conditions to see that:

Df .xı/ D �qı C Bxı C 4jxıj3 and D2f .xı/ � B C 12jxıj2:

Together with the ��semiconvexity of f , this provides:

Df .xı/ D O.ı/ and ��I � D2f .xı/ � B C O.ı2/: (6.28)

Clearly f inherits the twice differentiability of ' at xı . Then

�Df .xı/; D2f .xı/

� 2 J 2;Cf .xı/ \ J 2;�f .xı/;

and, in view of (6.28), we may send ı to zero along some subsequence and obtain alimit point .0; X/ 2 NJ 2;Cf .0/ \ NJ 2;�f .0/. utProof of Lemma 6.25. (i) Since v is bounded, there is a maximizer:

Ov�.x/ D v.y/ � �

2jx � yj2: (6.29)


By the definition of Ov� and the fact that .p; A/ 2 J 2;C Ov.x/, we have for everyx0; y0 2 R

N :

v.y0/ � �

2jx0 � y0j2 � Ov.x0/

� Ov.x/ C p � .x0 � x/ C 1

2A.x0 � x/ � .x0 � x/ C ı.x0 � x/

D v.y/ � �

2jx � yj2 C p � .x0 � x/

C 1

2A.x0 � x/ � .x0 � x/ C ı.x0 � x/; (6.30)

where we used (6.29) in the last equality.By first setting x0 D y0 C y � x in (6.30), we see that:

v.y0/ � v.y/ C p � .y0� y/ C 1

2A.y0 � y/ � .y0 � y/ C ı.y0 � y/ for all y0 2 R

N ;

which means that .p; A/ 2 J 2;Cy.y/.On the other hand, setting y0 D y in (6.30), we deduce that:

�.x0 � x/ ��x C x0

2C p

�� y

�� O

�jx � x0j2�;

which implies that y D x C p

�.

(ii) Consider a sequence .xn; pn; An/ with .xn; Ov�.xn/; pn; An/ �! .x; Ov�.x/; 0; A/

and .pn; An/ 2 J 2;C Ov�.xn/. In view of (i) and the definition of NJ 2;Cv.x/, itonly remains to prove that

v�xn C pn

�

��! v.x/: (6.31)

To see this, we use the upper semicontinuity of v together with (i) and thedefinition of Ov�:

v.x/ � lim supn

v�xn C pn

�

�

� lim infn

v�xn C pn

�

�

D limn

Ov�.xn/ C 1

2�jpnj2 D Ov�.x/ � v.x/: ut

Chapter 7Dynamic Programming Equationin the Viscosity Sense

7.1 DPE for Stochastic Control Problems

We now turn to the stochastic control problem introduced in Sect. 3.1. The chiefgoal of this section is to use the notion of viscosity solutions in order to relax thesmoothness condition on the value function V in the statement of Propositions 3.4and 3.5. Notice that the following proofs are obtained by slight modification of thecorresponding proofs in the smooth case.

Remark 7.1. Recall that the general theory of viscosity applies for nonlinear partialdifferential equations on an open domain O. This indeed ensures that the optimizerin the definition of viscosity solutions is an interior point. In the setting of controlproblems with finite horizon, the time variable moves forward so that the leftboundary of the time interval is not relevant. We shall then write the DPE on thedomain S D Œ0; T / � R

d . Although this is not an open domain, the general theoryof viscosity solutions is still valid.

We first recall the setting of Sect. 3.1. We shall concentrate on the finite horizoncase T < 1, while keeping in mind that the infinite horizon problems are handledby exactly the same arguments. The only reason why we exclude T D 1 isbecause we do not want to be diverted by issues related to the definition of theset of admissible controls.

Given a subset U of Rk , we denote by U the set of all progressively measurableprocesses � D f�t ; t < T g valued in U and by U0 WD U \ H

2. The elements of U0

are called admissible control processes.The controlled state dynamics is defined by means of the functions

b W .t; x; u/ 2 S � U �! b.t; x; u/ 2 Rn

and

� W .t; x; u/ 2 S � U �! �.t; x; u/ 2 MR.n; d/;


89

90 7 Dynamic Programming Equation in the Viscosity Sense

which are assumed to be continuous and to satisfy the conditions

jb.t; x; u/ � b.t; y; u/j C j�.t; x; u/ � �.t; y; u/j � K jx � yj; (7.1)

jb.t; x; u/j C j�.t; x; u/j � K .1 C jxj C juj/; (7.2)

for some constant K independent of .t; x; y; u/. For each admissible control process� 2 U0, the controlled stochastic differential equation:

dXt D b.t; Xt ; �t /dt C �.t; Xt ; �t /dWt; (7.3)

has a unique solution X , for all given initial data � 2 L2.F0;P/ with

E

"sup

0�s�t

jX�s j2

#< C.1 C EŒj�j2�/eC t for all t 2 Œ0; T �; (7.4)

for some constant C . Finally, the gain functional is defined via the functions:

f; k W Œ0; T / � Rd � U �! R and g W R

d �! R;

which are assumed to be continuous, kk�k1 < 1, and:

jf .t; x; u/j C jg.x/j � K.1 C juj C jxj2/;

for some constant K independent of .t; x; u/. The cost function J on Œ0; T � � Rd �

U is:

J.t; x; �/ WD E

�Z T

t

ˇ�.t; s/f .s; Xt;x;�s ; �s/ds C ˇ�.t; T /g

�Xt;x;�

T

��; (7.5)

when this expression is meaningful, where

ˇ�.t; s/ WD exp

��

Z s

t

k.r; Xt;x;�r ; �r/dr

�;

and fXt;x;�s ; s � tg is the solution of (7.3) with control process � and initial condition

Xt;x;�t D x. The stochastic control problem is defined by the value function:


J.t; x; �/ for .t; x/ 2 S: (7.6)

We recall the expression of the Hamiltonian

H.:; r; p; A/ WD supu2U

�f .:; u/ � k.:; u/r C b.:; u/ � p C 1

2Tr

��T.:; u/A

�; (7.7)

7.1 DPE for Stochastic Control Problems 91

and the second order operator associated to X and ˇ

Luv WD �k.:; u/v C b.:; u/ � Dv C 1

2Tr

��T.:; u/D2v

; (7.8)

which appears naturally in the following Ito’s formula valid for any smooth testfunction v:

dˇ�.0; t/v.t; X�t / D ˇ�.0; t/

.@t CL�t /v.t; X�

t /dt CDv.t; X�t / � �.t; X�

t ; �t /dWt

�:

Proposition 7.2. Assume that V is locally bounded on Œ0; T / �Rd . Then, the value

function V is a viscosity supersolution of the equation


� � 0 (7.9)

on Œ0; T / � Rd .

Proof. Let .t; x/ 2 S and ' 2 C 2.S/ be such that

0 D .V� � '/.t; x/ D minS

.V� � '/: (7.10)

Let .tn; xn/n be a sequence in S such that

.tn; xn/ �! .t; x/ and V.tn; xn/ �! V�.t; x/:

Since ' is smooth, notice that

�n WD V.tn; xn/ � '.tn; xn/ �! 0:

Next, let u 2 U be fixed, and consider the constant control process � D u. We shalldenote by Xn WD Xtn;xn;u the associated state process with initial data Xn

tnD xn.

Finally, for all n > 0, we define the stopping time:

�n WD inf fs > tn W .s � tn; Xns � xn/ 62 Œ0; hn/ � ˛Bg;

where ˛ > 0 is some given constant, B denotes the unit ball of Rn, and

hn WD p�n1f�n¤0g C n�11f�nD0g:

Notice that �n �! t as n �! 1.

1. From the first inequality in the dynamic programming principle of Theorem 3.3,it follows that:

0 � E

"V.tn; xn/ � ˇ.tn; �n/V�.�n; Xn

�n/ �

Z �n

tn

ˇ.tn; r/f .r; Xnr ; �r /dr

#:


Now, in contrast with the proof of Proposition 3.4, the value function is notknown to be smooth, and therefore we cannot apply Ito’s formula to V . Themain trick of this proof is to use the inequality V� � ' on S, implied by (7.10),so that we can apply Ito’s formula to the smooth test function ':

0 � �n C E

"'.tn; xn/ � ˇ.tn; �n/'.�n; Xn

�n/ �

Z �n

tn

ˇ.tn; r/f .r; Xnr ; �r /dr

#

D �n � E

"Z �n

tn

ˇ.tn; r/.@t ' C L�' � f /.r; Xnr ; u/dr

#

� E

"Z �n

tn

ˇ.tn; r/D'.r; Xnr /�.r; Xn

r ; u/dWr

#;

where @t ' denotes the partial derivative with respect to t .2. We now continue exactly along the lines of the proof of Proposition 3.5. Observe

that ˇ.tn; r/D'.r; Xnr /�.r; Xn

r ; u/ is bounded on the stochastic interval Œtn; �n�.Therefore, the second expectation on the right hand side of the last inequalityvanishes, and

�n

hn

� E

"1

hn

Z �n

tn

ˇ.tn; r/.@t ' C L�' � f /.r; Xr; u/dr

#� 0:

We now send n to infinity. The a.s. convergence of the random value insidethe expectation is easily obtained by the mean value theorem; recall thatfor n � N.!/ sufficiently large, �n.!/ D hn. Since the random variableh�1

n

R �n

t ˇ.tn; r/.L�' � f /.r; Xnr ; u/dr is essentially bounded, uniformly in n,

on the stochastic interval Œtn; �n�, it follows from the dominated convergencetheorem that:

�@t '.t; x/ � Lu'.t; x/ � f .t; x; u/ � 0;

which is the required result, since u 2 U is arbitrary. utWe next wish to show that V satisfies the nonlinear partial differential equa-

tion (7.9) with equality, in the viscosity sense. This is also obtained by a slightmodification of the proof of Proposition 3.5.

Proposition 7.3. Assume that the value function V is locally bounded on S. Letthe function H be finite and upper semicontinuous on Œ0; T / � R

d � Rd � Sd , and

kkCk1 < 1. Then, V is a viscosity subsolution of the equation


� � 0 (7.11)

on Œ0; T / � Rn.

7.1 DPE for Stochastic Control Problems 93

Proof. Let .t0; x0/ 2 S and ' 2 C 2.S/ be such that

0 D .V � � '/.t0; x0/ > .V � � '/.t; x/ for .t; x/ 2 S n f.t0; x0/g: (7.12)

In order to prove the required result, we assume to the contrary that

h.t0; x0/ WD @t '.t0; x0/ C H�t0; x0; '.t0; x0/; D'.t0; x0/; D2'.t0; x0/

�< 0

and work towards a contradiction.

1. Since H is upper semicontinuous, there exists an open neighborhood Nr WD.t0 � r; t0 C r/ � rB.t0; x0/ of .t0; x0/, for some r > 0, such that

h WD @t ' C H�:; '; D'; D2'

�< 0 on Nr : (7.13)

Then it follows from (7.12) that

� 2�erkkCk1 WD max@N�

.V � � '/ < 0: (7.14)

Next, let .tn; xn/n be a sequence in Nr such that

.tn; xn/ �! .t0; x0/ and V.tn; xn/ �! V �.t0; x0/:

Since .V � '/.tn; xn/ �! 0, we can assume that the sequence .tn; xn/ alsosatisfies:

j.V � '/.tn; xn/j � � for all n � 1: (7.15)

For an arbitrary control process � 2 Utn , we define the stopping time

��n WD infft > tn W X

tn;xn;�t 62 Nrg;

and we observe that��

n ; Xtn;xn;��

n

� 2 @Nr by the pathwise continuity of thecontrolled process. Then, with ˇ�

s WD ˇ�.tn; s/, it follows from (7.14) that

ˇ��

n'

��

n ; Xtn;xn;��

n

� � 2� C ˇ��

nV ��

��n ; X

tn;xn;��

n

�: (7.16)

2. Since ˇ�tn

D 1, it follows from (7.15) and Ito’s formula that

V.tn; xn/ � �� C '.tn; xn/

D �� C E

"ˇ�

��n'

��

n ; Xtn;xn;��

n

� �Z ��

n

tn

ˇ�s .@t C L�s /'

�s; Xtn;xn;�

s

�ds

#


� �� C E

"ˇ�

��n'

��

n ; Xtn;xn;��

n

�C Z ��n

tn

ˇ�s

�f .:; �s/ � h

��s; Xtn;xn;�

s

�ds

#

� �� C E

"ˇ�

��n'

��

n ; Xtn;xn;��

n

� CZ ��

n

tn

ˇ�s f

�s; Xtn;xn;�

s ; �s

�ds

#;

by (7.13). Using (7.16), this provides

V.tn; xn/ � � C E

"ˇ�

��nV ��

��n ; X

tn;xn;��

n

� CZ ��

n

tn

ˇ�s f

�s; Xtn;xn;�

s ; �s

�ds

#:

Since � > 0 does not depend on �, it follows from the arbitrariness of � 2 Utn

that latter inequality is in contradiction with the second inequality of the dynamicprogramming principle of Theorem 3.3. utAs a consequence of Propositions 7.3 and 7.2, we have the main result of this

section:

Theorem 7.4. Let the conditions of Propositions 7.3 and 7.2 hold. Then, the valuefunction V is a viscosity solution of the Hamilton-Jacobi-Bellman equation

� @t V � H��; V; DV; D2V

� D 0 on S: (7.17)

The partial differential equation (7.17) has a very simple and specific dependencein the time-derivative term. Because of this, it is usually referred to as a parabolicequation.

In order to a obtain a characterization of the value function by means ofthe dynamic programming equation, the latter viscosity property needs to becomplemented by a uniqueness result. This is usually obtained as a consequenceof a comparison result.

In the present situation, one may verify the conditions of Theorem 6.21. Forcompleteness, we report a comparison result which is adapted for the class ofequations corresponding to stochastic control problems.

Consider the parabolic equation

@t u C G�t; x; Du.t; x/; D2u.t; x/

� D 0 on S; (7.18)

where G is elliptic and continuous. For � > 0, set

GC� .t; x; p; A/ WD sup fG.s; y; p; A/ W .s; y/ 2 BS.t; xI �/g ;

G�� .t; x; p; A/ WD inf fG.s; y; p; A/ W .s; y/ 2 BS.t; xI �/g ;

where BS.t; xI �/ is the collection of elements .s; y/ in S such that jt �sj2 Cjx�yj2� �2. We report, without proof, the following result from [20] (Theorem V.8.1 andRemark V.8.1).

7.2 DPE for Optimal Stopping Problems 95

Assumption 7.5. The above operators satisfy

lim sup"&0

˚GC�".t"; x"; p"; A"/ � G��" .s"; y"; p"; B"/

�� Const .jt0 � s0j C jx0 � y0j/ Œ1 C jp0j C ˛ .jt0 � s0j C jx0 � y0j/� (7.19)

for all sequences .t"; x"/, .s"; y"/ 2 Œ0; T / � Rn, p" 2 R

n, and �" � 0 with:

..t"; x"/; .s"; y"/; p"; �"/ �! ..t0; x0/; .s0; y0/; p0; 0/ as " & 0;

and symmetric matrices .A"; B"/ with

�KI2n ��

A" 0

0 �B"

�� 2˛

�In �In

�In In

�

for some ˛ independent of ".

Theorem 7.6. Let Assumption 7.5 hold true, and let u 2 USC. NS/, and v 2 LSC. NS/

be viscosity subsolution and supersolution of (7.18), respectively. Then

supS

.u � v/ D supRn

.u � v/.T; �/:

A sufficient condition for (7.19) to hold is that f .�; �; u/; k.�; �; u/; b.�; �; u/; and�.�; �; u/ 2 C 1.S/ with

kbt k1 C kbxk1 C k�t k1 C k�xk1 < 1;

jb.t; x; u/j C j�.t; x; u/j � Const.1 C jxj C juj/I

see [20], Lemma V.8.1.

7.2 DPE for Optimal Stopping Problems

We first recall the optimal stopping problem considered in Sect. 4.1. For 0 � t �T � 1, the set TŒt;T � denotes the collection of all F�stopping times with values inŒt; T �. The state process X is defined by the SDE:

dXt D .t; Xt/dt C �.t; Xt /dWt; (7.20)

where and � are defined on NS WD Œ0; T / � Rn, take values in R

n and Sn,respectively, and satisfy the usual Lipschitz and linear growth conditions so that theabove SDE has a unique strong solution satisfying the integrability of Theorem 2.2.


For a measurable function g W Rn �! R, satisfying E�sup0�t<T jg.Xt /j

< 1,

the gain criterion is given by:

J.t; x; / WD E�g

�Xt;x

�1<1

for all .t; x/ 2 S; 2 TŒt;T �: (7.21)

Here, Xt;x denotes the unique strong solution of (4.1) with initial conditionX

t;xt D x. Then, the optimal stopping problem is defined by

V.t; x/ WD sup2TŒt;T �

J.t; x; / for all .t; x/ 2 S: (7.22)

The next result derives the dynamic programming equation for the previous optimalstopping problem in the sense of viscosity solution, thus relaxing the C 1;2 regularitycondition in the statement of Theorem 4.5. As usual, the same methodology allowsto handle seemingly more general optimal stopping problems:

NV .t; x/ WD sup2TŒt;T �

NJ .t; x; /; (7.23)

where

NJ .t; x; / WD E

�Z T

t

ˇ.t; s/f .s; Xt;xs /ds C ˇ.t; /g.Xt;x

/1f<1g�

;

ˇ.t; s/ WD exp

�Z s

t

k.u; Xt;xu /du

�:

Theorem 7.7. Assume that V is locally bounded, and let g W Rn �! R be

continuous. Then V is a viscosity solution of the obstacle problem

min f�.@t C A/V ; V � gg D 0 on S: (7.24)

Proof. (i) We first show that V is a viscosity supersolution. As in the proof ofTheorem 4.5, the inequality V � g � 0 is obvious, and implies that V� � g.Let .t0; x0/ 2 S and ' 2 C 2.S/ be such that

0 D .V� � '/.t0; x0/ D minS

.V� � '/:

To prove that �.@t C A/'.t0; x0/ � 0, we consider a sequence .tn; xn/n�1 �Œt0 � h; t0 C h� � B , for some small h > 0, such that

.tn; xn/ �! .t0; x0/ and V.tn; xn/ �! V�.t0; x0/:

Let .hn/n be a sequence of positive scalars converging to zero, to be fixed later,and introduce the stopping times:

�nhn

WD inf˚t > tn W .t; X

tn;xnt / 62 Œt0 � hn; t0 C hn� � B

�:

7.2 DPE for Optimal Stopping Problems 97

Then �hn 2 T tŒt;T � for sufficiently small h, and it follows from (4.10) that:

V.tn; xn/ � E�V�

��n

h ; X�nh

�:

Since V� � ', and denoting �n WD .V � '/.tn; xn/, this provides

�n C '.tn; xn/ � E�'

��n

h ; X�nh

�where �n �! 0:

We continue by fixing

hn WD p�n1f�n¤0g C n�11f�nD0g;

as in the proof of Proposition 7.2. Then, the rest of the proof follows exactlythe line of argument of the proof of Theorem 4.5 combined with that ofProposition 7.2.

(ii) We next prove that V is a viscosity subsolution of the (7.24). Let .t0; x0/ 2 Sand ' 2 C 2.S/ be such that

0 D .V � � '/.t0; x0/ D strict maxS

.V � � '/;

assume to the contrary that

.V � � g/.t0; x0/ > 0 and �.@t C A/'.t0; x0/ > 0;

and let us work towards a contradiction of the weak dynamic programmingprinciple.

Since g is continuous, and V �.t0; x0/ D '.t0; x0/, we may find constantsh > 0 and ı > 0 so that

' � g C ı and � .@t C A/' � 0 on Nh WD Œt0; t0 C h� � hB; (7.25)

where B is the unit ball centered at x0. Moreover, since .t0; x0/ is a strictmaximizer of the difference V � � ',

� � WD max@Nh

.V � � '/ < 0: (7.26)

Let .tn; xn/ be a sequence in S such that

.tn; xn/ �! .t0; x0/ and V.tn; xn/ �! V �.t0; x0/:

We next define the stopping times:

�n WD inf˚t > tn W �

t; Xtn;xnt

� 62 Nh

�;


and we continue as in Step 2 of the proof of Theorem 4.5. We denote �n WDV.tn; xn/ � '.tn; xn/, and we compute by Ito’s formula that for an arbitrarystopping rule 2 T t

Œt;T �,

V.tn; xn/ D �n C '.tn; xn/

D �n C E

"' . ^ �n; X^�n/ �

Z ^�n

tn

.@t C A/'.t; Xt /dt

#;

where diffusion term has zero expectation because the process .t; Xtn;xnt / is

confined to the compact subset Nh on the stochastic interval Œtn; ^ �n�. Since�.@t C A/' � 0 on Nh by (7.25), this provides

V.tn; xn/ � �n C E�' .; X/ 1f<�ng C ' .�n; X�n/ 1�f�ngC

� E

�.g .X/ C ı/ 1f<�ng C �

V � .�n; X�n/ C ��

1f�n�g

� � ^ ı C E�g .X/ 1f<�ng C V � .�n; X�n/ 1f�n�g

;

where we used the fact that ' � g C ı on Nh by (7.25), and ' � V � C � on@Nh by (7.26). Since �n WD .V � '/.tn; xn/ �! 0 as n ! 1, and 2 T t

Œt;T � isarbitrary, this provides the desired contradiction of (4.9). ut

7.3 A Comparison Result for Obstacle Problems

In this section, we derive a comparison result for the obstacle problem:

min˚F.:; u; @t u; Du; D2u/; u � g

� D 0 on Œ0; T / � Rd (7.27)

The dynamic programming equation of the optimal stopping problem (7.23)corresponds to the particular case:

F.:; u; @t u; Du; D2u/ D @t u C b � Du C 1

2Tr

��TD2u

� ku C f:

Theorem 7.8. Let F be a uniformly continuous elliptic operator satisfyingAssumption 6.20. Let u 2 USC.O/ and v 2 LSC.O/ be viscosity subsolutionand supersolution of the equation (7.27), respectively, with sub-quadratic growth.Then

u.T; :/ � v.T; :/ H) u � v on Œ0; T � � Rd :

Proof. This is an easy adaptation of the proof of Theorem 6.21. We adapt the samenotations so that, in the present, x stands for the pair .t; x/. The only difference

7.3 A Comparison Result for Obstacle Problems 99

appears at Step 3 which starts from the fact that

min fF.xn; u.xn/; ˛n.xn � yn/ C "xn; An/; u.xn/ � g.xn/g � 0;

min fF.yn; v.yn/; ˛n.xn � yn/ � "yn; Bn/; v.yn/ � g.yn/g � 0:

This leads to two cases:

– Either u.xn/ � g.xn/ � 0 along some subsequence. Then the inequality v.yn/ �g.yn/ � 0 leads to a contradiction of (6.19).

– Or F.xn; u.xn/; ˛n.xn � yy/ C "xn; An/ � 0, which can be combined with thesupersolution part F.yn; v.yn/; ˛n.xn � yy/ � "yn; Bn/ � 0 exactly as in theproof of Theorem 6.21, and leads to a contradiction of (6.19). ut

Chapter 8Stochastic Target Problems

8.1 Stochastic Target Problems

In this section, we study a special class of stochastic target problems which avoidsfacing some technical difficulties, but reflects in a transparent way the main ideasand arguments to handle this new class of stochastic control problems.

All of the applications that we will be presenting fall into the framework of thissection. The interested readers may consult the references at the end of this chapterfor the most general classes of stochastic target problems, and their geometricformulation.

8.1.1 Formulation

Let T > 0 be the finite time horizon andW D fWt , 0 � t � T g be a d -dimensionalBrownian motion defined on a complete probability space .˝;F ;P/. We denote byF D fFt , 0 � t � T g the P-augmentation of the filtration generated by W .

We assume that the control set U is a convex compact subset of Rd with

nonempty interior, and we denote by U the set of all progressively measurableprocesses � D f�t ; 0 � t � T g with values in U .

The state process is defined as follows: given the initial data z D .x; y/ 2 Rd�R,

an initial time t 2 Œ0; T �, and a control process � 2 U , let the controlled processZt;z;� D .Xt;x;� ; Y t;z;�/ be the solution of the stochastic differential equation:

dXt;x;�r D �

�r; Xt;x;�

r ; �r�

dr C ��r; Xt;x;�

r ; �r�

dWr;

dY t;z;�r D b�r;Zt;z;�

r ; �r�

dr C �r � dWr; r 2 .t; T /;

with initial data

Xt;x;�t D x; and Y t;x;y;�t D y:


101

102 8 Stochastic Target Problems

Here,� W S�U �! Rd , � W S�U �! Sd , and b W S�R�U �! R are continuous

functions, Lipschitz in .x; y/ uniformly in .t; u/. Then, all above processes are welldefined for every admissible control process � 2 U0 defined by

U0 WD�� 2 U W E

�Z t

0

�j�0.s; �s/jCjb0.s; �s/j C j�0.s; �s/j2 C j�sj2�

ds

�< 1

�;

where �0.t; u/ WD �.t; 0; u/, b0.t; u/ WD b.t; 0; u/, and �0.t; u/ WD �.t; 0; u/.Throughout this section, we assume that the function

u 7�! �.t; x; u/p

has a unique fixed point for every .t; x/ 2 NS � R defined by

�.t; x; u/p D u ” u D .t; x; p/: (8.1)

For a measurable function g W Rd �! R, we define the stochastic target problem by

V.t; x/ WD inf˚y 2 R W Y t;x;y;�T � g

�Xt;x;�T

�; P � a.s. for some � 2 U0

�: (8.2)

Remark 8.1. By introducing the subset of control processes

A.t; x; y/ WD ˚� 2 U0 W Y t;x;y;�T � g

�Xt;x;�T

�; P � a.s.

�;

we may re-write the value function of the stochastic target problem into

V.t; x/ D infY.t; x/; where Y.t; x/ WD fy 2 R W A.t; x; y/ ¤ ;g :

The set Y.t; x/ satisfies the following important property

for all y 2 R; y 2 Y.t; x/ H) Œy;1/ � Y.t; x/:

Indeed, since the state process Xt;x;� is independent of y, the process Y t;x;y;� isa solution of a stochastic differential equation (with random coefficients), and thecorresponding flow y 7�! Y

t;x;y;�t is increasing for every t by classical results on

SDEs.

8.1.2 Geometric Dynamic Programming Principle

Similar to the standard class of stochastic control and optimal stopping problemsstudied in the previous chapters, the main tool for the characterization of the valuefunction and the solution of stochastic target problems is the dynamic programming

8.1 Stochastic Target Problems 103

principle. Although the present problem does not fall into the class of problemsstudied in the previous chapters, the idea of dynamic programming is the same:allow the time origin to move, and deduce a relation between the value function atdifferent points in time.

In these notes, we shall essentially use the easy direction of a more generalgeometric dynamic programming principle. The geometric nature of this result willbe justified in Remark 8.4.

Theorem 8.2. Let .t; x/ 2 Œ0; T � � Rd and y 2 R such that A.t; x; y/ ¤ ;. Then,

for any control process � 2 A.t; x; y/ and stopping time � 2 TŒt;T �,

Y t;x;y;�� V��;Xt;x;�

�

�; P � a.s. (8.3)

Proof. Let z D .x; y/ and � 2 A.t; z/, and denote Zt;z;� WD .Xt;x;� ; Y t;z;�/. ThenYt;z;�T � g

�Xt;x;�T

�P�a.s. Notice that

Zt;z;�T D Z

�;Zt;z;�� ;�

T :

Then, by the definition of the set A, it follows that � 2 A��;Zt;z;�

�

�, and therefore

V��;Xt;x;�

�

� � Y t;z;�� , P�a.s. utIn the next subsection, we will prove that the value function V is a viscosity

supersolution of the corresponding dynamic programming equation which will beobtained as the infinitesimal counterpart of (8.3). The following remark commentson the full geometric dynamic programming principle in the context of stochastictarget problems. The proof of this claim is beyond the scope of these notes, and weshall only report a sketch of it.

Remark 8.3. The statement (8.3) in Theorem 8.2 can be strengthened to thefollowing geometric dynamic programming principle:

V.t; x/ D infny 2 R W Y t;x;y;�� V

��;Xt;x;�

�

�; P � a.s. for some � 2 U0

o; (8.4)

for all stopping time � with values in Œt; T �. Let us provide an intuitive justificationof this. Denote Oy WD V.t; x/. In view of (8.3), it is easily seen that (8.4) is implied by

P

hY t;x; Oy��;�� > V

��;Xt;x;�

�

�i< 1 for all � 2 U0 and � > 0:

In words, there is no control process � which allows to reach the value functionV.�;Xt;x;�

� / at time � , with full probability, starting from an initial data strictlybelow the value function V.t; x/. To see this, suppose to the contrary that thereexist � 2 U0, � > 0 and � 2 TŒt;T � such that:

Y t;x; Oy��;�� > V

��;Xt;x;�

�

�; P � a.s.


In view of Remark 8.1, this implies that Y t;x; Oy��;�� 2 Y

��;Xt;x;�

�

�, and therefore

there exists a control O� 2 U0 such that

Y�;Z

t;x; Oy��;�� ;O�

T � gX�;X

t;x;�� ;O�

T

; P � a.s.

Since the processX�;X

t;x;�� ;O� ; Y �;Z

t;x; Oy��;�� ;O�

depends on O� only through its realiza-

tions in the stochastic interval Œ�; T �, we may chose O� so as O� D � on Œt; � � (this is

the difficult part of this proof). Then Z�;Zt;x; Oy��;�� ;O�

T D Zt;x; Oy��;O�T and therefore Oy � �

2 Y.t; x/, hence V.t; x/ � Oy � �. This is the required contradiction as Oy D V.t; x/

and � > 0.

Remark 8.4. An extended version of the stochastic target problem that was intro-duced in [36] avoids the decoupling of the components Z D .X; Y /. In this case,there is no natural direction to isolate in the process Z which we assume defined bythe general dynamics:

dZt;z;�r D ˇ.r;Zt;z;�

r ; �r /dr C ˇ.r;Zt;z;�r ; �r/dWr; r 2 .t; T /:

The stochastic target problem is defined by the value function:

V.t/ WD ˚z 2 R

dC1 W Zt;z;�T 2 �; P � a.s.

�;

for some given target subset � � RdC1. Notice that V.t/ is a subset in R

dC1. It wasproved in [36] that for all stopping time � with values in Œt; T �:

V.t/ D inf˚z 2 R

dC1 W Zt;z;�� 2 V.�/; P � a.s.

�:

This is a dynamic programming principle for the sets fV.t/; t 2 Œ0; T �g, and for thisreason it was called geometric dynamic programming principle.

8.1.3 The Dynamic Programming Equation

In order to have a simpler statement and proof of the main result, we assume in thissection that

U is a closed convex subset of Rd ; int.U / ¤ ; and 0 2 U: (8.5)

The formulation of the dynamic programming equation involves the notion ofsupport function from convex analysis.

Dual Characterization of Closed Convex Sets We first introduce the supportfunction of the set U :

ıU ./ WD supx2U

x � ; for all 2 Rd :


By definition ıU is a convex function on Rd . Since 0 2 U , its effective domain is

given by

QU WD dom.ıU / D f 2 Rd W ıU ./ < 1g;

and is a closed convex cone of Rd . Since U is closed and convex by (8.5), we havethe following dual characterization:

x 2 U if and only if ıU ./ � x � � 0 for all 2 QU ; (8.6)

see, e.g., Rockafellar [35]. Moreover, since QU is a cone, we may normalize the dualvariables on the right hand side:

x 2 U if and only if ıU ./�x � � 0 for all 2 QU1 WD f 2 QU W jj D 1g: (8.7)

This normalization will be needed in our analysis in order to obtain a dualcharacterization of int.U /. Indeed, since U has nonempty interior by (8.5), we have

x 2 int.U / if and only if inf2 QU1 ıU ./ � x � > 0: (8.8)

Formal Derivation of the DPE We start with a formal derivation of the dynamicprogramming equation which provides the main intuitions.

To simplify the presentation, we suppose that the value function V is smooth andthat existence holds, i.e., for all .t; x/ 2 S, there is a control process O� 2 U0 suchthat, with z D .x; V .t; x//, we have Y t;z;O�T � g.X

t;x;O�T /, P�a.s. Then it follows from

the geometric dynamic programming of Theorem 8.2 that P�a.s:

Y t;z;�tCh D v.t; x/CZ tCh

t

bs; Zt;z;O�

s ; O�s

ds CZ tCh

t

O�s � dWs � Vt C h;Xt;x;O�

tCh:

By Ito’s formula, this implies that

0 �Z tCh

t

n�@tV .s; Xt;x;O�

s /

CHs; Zt;z;O�

s ;DV.s;Xt;x;O�s /;D2V.s;Xt;x;O�

s /; O�so

ds

CZ tCh

t

N �ss; Xt;x;O�

s ;DV.s;Xt;x;O�s /

� dWs; (8.9)

where we introduced the functions:

H.t; x; y; p;A; u/ WD b.t; x; y; u/ � �.t; x; u/ � p � 1

2Tr��.t; x; u/2A

�; (8.10)

N u.t; x; p/ WD u � �.t; x; u/p: (8.11)


We continue our intuitive derivation of the dynamic programming equation byassuming that all terms inside the integrals are bounded (we know that this can beachieved by localization). Then the first integral behaves like Ch, while the secondintegral can be viewed as a time-changed Brownian motion. By the properties of theBrownian motion, it follows that the integrand of the stochastic integral term mustbe zero at the origin:

N�tt .t; x;DV.t; x// D 0 or, equivalently; �t D

�t; x;DV.t; x/

�;

where was introduced in (8.1). In particular, this implies that

.t; x;DV.t; x// 2 U;or, equivalently,

ıU ./ � � .t; x;DV.t; x// � 0 for all 2 QU1; (8.12)

by (8.7). Taking expected values in (8.9), normalizing by h, and sending h to zero,we see that

�@tV .t; x/CH�t; x; V .t; x/;DV.t; x/;D2V.t; x/; .t; x;DV.t; x//

� � 0:

(8.13)

Putting (8.12) and (8.13) together, we obtain

min

(

�@tV CH�:; V;DV;D2V; .:;DV /

�; inf2 QU1

�ıU ./ � � .:;DV /�

)

� 0:

By using the second part of the geometric dynamic programming principle, seeRemark 8.3, we expect to prove that equality holds in the latter dynamic program-ming equation.

The Dynamic Programming Equation We next turn to a rigorous derivation ofthe dynamic programming equation. In the subsequent proof, we shall use thefirst part of the dynamic programming reported in Theorem 8.2 to prove thatthe stochastic target problem is a supersolution of the corresponding dynamicprogramming equation. For completeness, we will also provide the proof ofthe subsolution property based on the full dynamic programming principle ofRemark 8.3. We observe however that our subsequent applications will only makeuse of the supersolution property.

Theorem 8.5. Assume that V is locally bounded, and let the maps H and

be continuous. Then V is a viscosity supersolution of the dynamic programmingequation on S:

min

(

�@tV CH�:; V;DV;D2V; .:;DV /

�; inf2 QU1

�ıU ./ � � .:;DV /�

)

D 0:


Assume further that is locally Lipschitz continuous and U has nonempty interior.Then V is a viscosity solution of the above DPE on S.

Proof. As usual, we prove separately the supersolution and the subsolutionproperties.

1. Supersolution: Let .t0; x0/ 2 S and ' 2 C2.S/ be such that

.strict/minS.V� � '/ D .V� � '/.t0; x0/ D 0;

and assume to the contrary that

� 2� WD ��@tV CH�:; V;DV;D2V; .:;DV /

��.t0; x0/ < 0: (8.14)

(i) By the continuity of H and , we may find " > 0 such that

� @tV .t; x/CH�t; x; y;DV.t; x/;D2V.t; x/; u

� � ��for .t; x/ 2 B".t0; x0/; jy � '.t; x/j � "; and u 2 U s.t. jN u.t; x; p/j � ":

(8.15)

Notice that (8.15) is obviously true if fu 2 U W jN u.t; x; p/j � "g D ;, sothat the subsequent argument holds in this case as well.

Since .t0; x0/ is a strict minimizer of the difference V� � ', we have

WD min@B".t0;x0/

.V� � '/ > 0: (8.16)

(ii) Let .tn; xn/n � B".t0; x0/ be a sequence such that

.tn; xn/ �! .t0; x0/ and V.tn; xn/ �! V�.t0; x0/; (8.17)

and set yn WD V.tn; xn/ C n�1 and zn WD .xn; yn/. By the definition ofthe problem V.tn; xn/, there exists a control process O�n 2 U0 such that theprocessZn WD Ztn;zn;O�n satisfies Y nT � g.Xn

T /, P�a.s. Consider the stoppingtimes:

�0n WD inf ft > tn W .t; Xnt / 62 B".t0; x0/g ;

�n WD �0n ^ inf ft > tn W jY nt � '.t; Xnt /j � "g :

Then, it follows from the geometric dynamic programming principle that

Y nt^�n � V�t ^ �n;Xn

t^�n�:


Since V � V� � ', and using (8.16) and the definition of �n, this impliesthat

Y nt^�n � '�t ^ �n;Xn

t^�n�C 1ftD�ng

�1f�nD�0ng C "1f�n<�0ng

�

� '�t ^ �n;Xn

t^�n�C . ^ "/1ftD�ng: (8.18)

(iii) Denoting cn WD V.tn; xn/ � '.tn; xn/ � n�1, we write the process Y n as

Y nt D cn C '.tn; xn/CZ t

tn

b.s; Zns ; O�ns /ds C

Z t

tn

O�ns � dWs:

Plugging this into (8.18) and applying Ito’s formula, we then see that:

." ^ /1ftD�ng � cn CZ t^�n

tn

ıns ds

CZ t^�n

tn

N O�ns �s; Xns ;D'.s;X

ns /� � dWs

� Mn WD cn CZ t^�n

tn

ıns 1An.s/ds (8.19)

CZ t^�n

tn

N O�ns �s; Xns ;D'.s;X

ns /� � dWs;

where

ıns WD �@t'.s; Xns /CH

�s; Zn

s ;D'.s;Xns /;D

2'.s;Xns /; O�s

�

and

An WD fs 2 Œtn; �n� W ıns > ��g :

By (8.15), observe that the diffusion term �ns WD N O�ns �s; Xns ;D'.s;X

ns /�

in (8.19) satisfies j�ns j � � for all s 2 An. Then, by introducing theexponential local martingale Ln defined by

Lntn D 1 and dLnt D Lnt j�nt j�2�nt � dWt; t � tn;

we see that the processMnLn is a positive local martingale. ThenMnLn isa supermartingale, and it follows from (8.19) that

" ^ � E�Mn�nLn�n

� � MntnLntn D cn;

which can not happen because cn �! 0. Hence, our starting point (8.14)can not happen, and the proof of the supersolution property is complete.


2. Subsolution: Let .t0; x0/ 2 S and ' 2 C2.S/ be such that

.strict/maxS.V � � '/ D .V � � '/.t0; x0/ D 0; (8.20)

and assume to the contrary that

2� WD ��@t' CH�:; ';D';D2'; .:; '/

��.t0; x0/ > 0;

and inf2 QU1

�ıU ./ � � .:;D'/�.t0; x0/ > 0: (8.21)

(i) By the continuity of H and , and the characterization of int.U / in (8.8), itfollows from (8.21) that

��@t' CH�:; y;D';D2'; .:;D'/

�� and .:;D'/ 2 Ufor .t; x/ 2 B".t0; x0/ and jy � '.t; x/j � ": (8.22)

Also, since .t0; x0/ is a strict maximizer in (8.20), we have

� � WD max@pB".t0;x0/

.V � � '/ < 0; (8.23)

where @pB".t0; x0/ WD ft0 C "g � cl .B".t0; x0// [ Œt0; t0 C "/ � @B".x0/

denotes the parabolic boundary of B".t0; x0/.(ii) Let .tn; xn/n be a sequence in S which converges to .t0; x0/ and such that

V.tn; xn/ ! V �.t0; x0/. Set yn D V.tn; xn/ � n�1 and observe that

n WD yn � '.tn; xn/ �! 0: (8.24)

Let Zn WD .Xn; Y n/ denote the controlled state process associated to theMarkovian control O�nt D .t; Xn

t ;D'.t; Xnt // and the initial condition

Zntn

D .xn; yn/. Since is locally Lipschitz-continuous, the process Zn

is well defined. We next define the stopping times

�0n WD inf fs � tn W .s; Xns / … B".t0; x0/g ;

�n WD �0n ^ inf fs � tn W jY n.s/ � '.s;Xns /j � "g :

By the first line in (8.22), (8.24) and a standard comparison theorem, itfollows that Y n�n � '.�n;X

n�n/ � " on fjY n�n � '.�n;X

n�n/j � "g for n large

enough. Since V � V � � ', we then deduce from (8.23) and the definitionof �n that

Y n�n � V ��n;Xn�n

� � 1f�n<�0ngY n�n � '

��n;X

n�n

�

C1f�nD�0ngY n�0n

� V ��0n; Xn�0n

�


� "1f�n<�0ng C 1f�nD�ongY n�0n

� V ��0n; Xn�0n

�

� "1f�n<�0ng C 1f�nD�0ngY n�0n

C � � '��0n; X

n�0n

�

� " ^ � C 1f�nD�0ngY n�0n

� '��0n ; Xn�0n

�:

We continue by using Ito’s formula:

Y n�n � V��n;X

n�n

� � " ^ � C 1f�nD�0ng

n CZ �n

tn

˛.s; Xns ; Y

ns /ds

!

;

where the drift term ˛.�/ � � is defined in (8.22) and the diffusion coefficientvanishes by the definition of the function in (8.1). Since "; � > 0, andn ! 0, this implies that

Y n�n � V��n;X

n�n

�for sufficiently large n:

Recalling that the initial position of the process Y n is yn D V.tn; xn/ �n�1 < V.tn; xn/, this is clearly in contradiction with the second part of thegeometric dynamic programming principle discussed in Remark 8.3. ut

8.1.4 Application: Hedging Under Portfolio Constraints

As an application of the previous results, we now study the problem of superhedgingunder portfolio constraints in the context of the Black–Scholes model.

Formulation We consider a financial market consisting of d C 1 assets. The firstasset X0 is nonrisky and is normalized to unity. The d next assets are risky withprice process X D .X1; : : : ; Xd /T defined by the Black–Scholes model:

dXt D Xt ? �dWt ;

where � is a constant symmetric nondegenerate matrix in Rd , and x?� is the square

matrix in Rd with entries .x ? �/i;j D xi�i;j .

Remark 8.6. We observe that the normalization of the first asset to unity does notentail any loss of generality as we can always reduce to this case by discounting or,in other words, by taking the price process of this asset as a numeraire.

Also, the formulation of the above process X as a martingale is not a restrictionas our subsequent superhedging problem only involves the underlying probabilitymeasure through the corresponding zero-measure sets. Therefore, under the no-arbitrage condition (or more precisely, no free lunch with vanishing risk), we canreduce the model to the above martingale case by an equivalent change of measure.


Under the self-financing condition, the liquidation value of the portfolio isdefined by the controlled state process:

dY t D � t � dWt ;

where is the control process, with it representing the amount invested in the i thrisky asset Xi at time t .

We introduce portfolio constraints by imposing that the portfolio process mustbe valued in a subset U of Rd . We shall assume that

U is closed convex subset of Rd ; int.U / ¤ ;; and 0 2 U: (8.25)

We then define the controls set by Uo as in the previous sections, and we define thesuperhedging problem under portfolio constraints by the stochastic target problem:

V.t; x/ WD inf˚y W Y t;y; T � g.X

t;xT /; P � a.s. for some 2 U0

�; (8.26)

where g W RdC �! RC is a nonnegative LSC function with linear growth.We shall provide an explicit solution of this problem by only using the superso-

lution claim from Theorem 8.5. This will provide a minorant of the superhedgingcost V . To prove that this minorant is indeed the desired value function, we will usea verification argument.

Deriving a Minorant of the Superhedging Cost First, since 0 � g.x/ � C.1Cjxj/ for some constant C > 0, we deduce that 0 � V � C.1 C jxj/, the righthand side inequality is easily justified by the buy-and-hold strategy suggested by thelinear upper bound. Then, by a direct application of the first part of Theorem 8.5,we know that the LSC envelope V� of V is a supersolution of the DPE:

�@tV� � 1

2Tr�.x ? �/2D2V�

� � 0; (8.27)

ıU ./ � � .x ? DV�/� � 0; for all 2 QU : (8.28)

Notice that (8.28) is equivalent to

the map � 7�! h.�/ WD �ıU ./ � V�.t; x ? e� / is that nondecreasing, (8.29)

where e� is the vector of Rd with entries .e�/i D e�i . Then h.1/ � h.0/ provides

V�.t; x/ � sup2 QU

V��x ? e

� � ıU ./:

We next observe that V�.T; :/ � g (just use the definition of V , and send t % T ).Then, we deduce from the previous inequality that

V�.T; x/ � Og.x/ WD sup2 QU

g�x ? e

� � ıU ./ for all x 2 RdC: (8.30)


In other words, in order to superhedge the derivative security with final payoffg.XT /, the constraints on the portfolio require that one hedges the derivativesecurity with larger payoff Og.XT /. The function Og is called the face-lifted payoff,and is the smallest majorant of g which satisfies the gradient constraint x?Dg.x/ 2U for all x 2 R

dC.Combining (8.30) with (8.27), it follows from the comparison result for the linear

Black–Scholes PDE that

V.t; x/ � V�.t; x/ � v.t; x/ WD E� Og.Xt;x

T /�

for all .t; x/ 2 S: (8.31)

Explicit Solution Our objective is now to prove that V D v. To see this, considerthe Black–Scholes hedging strategy O of the derivative security Og.Xt;x

T /:

v.t; x/CZ T

t

O s � �dWs D Og.Xt;xT /:

Since Og has linear growth, it follows that O 2 H2. We also observe that the random

variable lnXt;xT is gaussian, so that the function v can be written in

v.t; x/ DZ

Og.ew/1

p2 �2.T � t/e

� 12

w�xC

12 �2.T�t /

�p

T�t

�2

dw:

Under this form, it is clear that v is a smooth function. Then the above hedgingportfolio is given by

O s WD Xt;xs ? DV.s;Xt;x

s /:

Notice that, for all 2 QU ,

�ıU ./ � v.t; xe� / D E

h�ıU ./ � Og

Xt;xe�

T

i

is nondecreasing in � by applying (8.29) to Og which, by definition, satisfies x ?Dg.x/ 2 U for all x 2 R

dC. Then, x ? Dg.x/ 2 U , and therefore the abovereplicating portfolio O takes values in U . Since Og � g, we deduce from (8.30) thatv � V .

8.2 Stochastic Target Problem with Controlled Probabilityof Success

In this section, we extend the above problem to the case where the target has to bereached only with a given probability p:

OV .t; x; p/ WD inf˚y 2 RC W P �Y t;x;y;�T � g.X

t;x;�T /

� � p for some � 2 U0�:

(8.32)

8.2 Stochastic Target Problem with Controlled Probability of Success 113

In order to avoid degenerate results, we restrict the analysis to the case where the Yprocess takes non-negative values, by simply imposing the following conditions onthe coefficients driving its dynamics:

b.t; x; 0; u/ � 0 for all.t; x/ 2 S; u 2 U: (8.33)

Notice that the above definition implies that

0 D OV .:; 0/ � OV � OV .:; 1/ D V; (8.34)

and

OV .:; p/ D 0 for p < 0 and OV .:; p/ D 1 for p > 1 ; (8.35)

with the usual convention inf ; D 1.

8.2.1 Reduction to a Stochastic Target Problem

Our first objective is to convert this problem into a (standard) stochastic targetproblem, so as to apply the geometric dynamic programming arguments of theprevious section.

To do this, we introduce an additional controlled state variable:

P t;p;˛s WD p C

Z s

t

˛r � dWr; for s 2 Œt; T �; (8.36)

where the additional control ˛ is an F�progressively measurable Rd�valued

process satisfying the integrability condition E

hR T0

j˛sj2dsi< 1. We then set

OX WD .X; P /, OS WD Œ0; T / � Rd � .0; 1/, OU WD U � R

d , and denote by OU thecorresponding set of admissible controls. Finally, we introduce the function:

G. Ox; y/ WD 1fy�g.x/g � p for y 2 R; Ox WD .x; p/ 2 Rd � Œ0; 1�:

Proposition 8.7. For all t 2 Œ0; T � and Ox D .x; p/ 2 Rd � Œ0; 1�, we have

OV .t; Ox/ D infny 2 RC W G

OXt; Ox;O�T ; Y

t;x;y;�T

� 0 for some O� D .�; ˛/ 2 OU

o:

Proof. We denote by v.t; x; p/ the value function appearing on the right hand.We first show that OV � v. For y > OV .t; x; p/, we can find � 2 U such

that p0 WD P�Yt;x;y;�T � g

�Xt;x;�T

�� p. By the stochastic integral representationtheorem, there exists an F-progressively measurable process ˛ such that

1fY t;x;y;�T �g.Xt;x;�T /g D p0 CZ T

t

˛s � dWs D Pt;p0;˛T and E

�Z T

t

j˛sj2ds�< 1:


Since p0 � p, it follows that 1fY t;x;y;�T �g.Xt;x;�T ;/g � Pt;p;˛T , and therefore y �

v.t; x; p/ from the definition of the problem v.We next show that v � OV . For y > v.t; x; p/, we have G

� OXt; Ox;O�T ; Y

t;x;y;�T

� � 0

for some O� D .�; ˛/ 2 OU . Since P˛t;p is a martingale, it follows that

P�Yt;x;y;�T � g

�Xt;x;�T

�� D E

h1fY t;x;y;�T �g.Xt;x;�T /g

i� E

�Pt;p;˛T

� D p;

which implies that y � OV .t; x; p/ by the definition of OV . utRemark 8.8. 1. Suppose that the infimum in the definition of OV .t; x; p/ is achieved

and there exists a control � 2 U0 satisfying P�Yt;x;y;�T � g

�Xt;x;�T

�� D p, theabove argument shows that:

P t;p;˛s D P

hYt;x;y;�T � g

�Xt;x;�T

� ˇˇˇFs

ifor all s 2 Œt; T �:

2. It is easy to show that one can moreover restrict to control ˛ such that the processP t;p;˛ takes values in Œ0; 1�. This is rather natural since this process shouldbe interpreted as a conditional probability, and this corresponds to the naturaldomain Œ0; 1� of the variable p. We shall however avoid to introduce this stateconstraint, and use the fact that the value function OV .�; p/ is constant for p � 0

and equal 1 for p > 1, see (8.35).


The above reduction of the problem OV to a stochastic target problem allows to applythe geometric dynamic programming principle of the previous section, and to derivethe corresponding dynamic programming equation. For Ou D .u; ˛/ 2 OU and Ox D.x; p/ 2 R

d � Œ0; 1�, set

O�. Ox; Ou/ WD �.x; u/0

�; O�. Ox; Ou/ WD

�.x; u/˛T

�:

For .y; q; A/ 2 R � RdC1 � SdC1 and Ou D .u; ˛/ 2 OU ,

ON Ou.t; Ox; y; q/ WD u � O�.t; Ox; Ou/q D N u.t; x; qx/� qp˛ for q D .qx; qp/2Rd�R;

and we assume that

u 7�! N u.t; x; qx/ is one to one, with inverse function .t; x; qx/: (8.37)


Then, by a slight extension of Theorem 8.5, the corresponding dynamicprogramming equation is given by

0 D �@t OV C sup˛

nb.:; OV ; .:;Dx

OV ; ˛DpOV // � �.:; .:;Dx

OV ; ˛DpOV //:Dx

OV

�12

Trh�.:; .:;Dx

OV ; ˛DpOV //2D2

xOVi

�12˛2D2

pOV � ˛�.:; .:;Dx

OV ; ˛DpOV //Dxp

OVo:

The application in the subsequent section will be only making use of the supersolu-tion property of the stochastic target problem.

8.2.3 Application: Quantile Hedging in the Black–ScholesModel

The problem of quantile hedging was solved by Follmer and Leukert [21] in thegeneral model of asset prices process (non-necessarily Markovian), by means ofthe Neyman–Pearson lemma from mathematical statistics. The stochastic controlapproach developed in the present section allows to solve this type of problems ina wider generality. The objective of this section is to recover the explicit solutionof [21] in the context of a complete financial market where the underlying riskyasset prices are not affected by the control:

�.x; u/ D �.x/ and �.x; u/ D �.x/ are independent of u, (8.38)

where � and � are Lipschitz-continuous, and �.x/ is invertible for all x.Notice that we will be only using the supersolution property from the results of

the previous sections.

The Financial Market The process X , representing the price process of d riskyassets, is defined by Xt;x

t D x 2 .0;1/d , and

dXt;xs D Xt;x

s ? �.Xt;xs /

��.Xt;x

s /ds C dWs

�,where � WD ��1�:

We assume that the coefficients � and � are such that Xt;x 2 .0;1/d P�a.s. forall initial conditions .t; x/ 2 Œ0; T � � .0;1/d . In order to avoid arbitrage, we alsoassume that � is invertible and that

supx2.0;1/d

j�.x/j < 1: (8.39)


The drift coefficient of the controlled process Y is given by:

b.t; x; y; u/ D u � �.x/: (8.40)

The control process � is valued in U D Rd , with components �is indicating the

dollar investment in the i -th security at time s. After the usual reduction of theinterest rates to zero, it follows from the self-financing condition that the liquidationvalue of the portfolio is given by

Y t;x;y;�s D y CZ s

t

�r � �.Xt;xs /

��.Xt;x

s /ds C dWs

�; s � t ;

The Quantile Hedging Problem The quantile hedging problem of the derivativesecurity g.Xt;x

T / is defined by the stochastic target problem with controlled proba-bility of success:

OV .t; x; p/ WD inf˚y 2 RC W P

�Yt;x;y;�T � g.X

t;xT /

� � p for some � 2 U0�:

We shall assume throughout that 0 � g.x/ � C.1 C jxj/ for all x 2 RdC. By the

usual buy-and-hold hedging strategies, this implies that 0 � V.t; x/ � C.1C jxj/.Under the above assumptions, the corresponding super hedging cost V.t; x/ WD

OV .t; x; 1/ is continuous and is given by

V.t; x/ D EQt;x

�g.Xt;x

T /�;

where Qt;x is the P-equivalent martingale measure defined by

dQt;x

dPD exp

�12

Z T

t

j�.Xt;xs /j2ds �

Z T

t

�.Xt;xs / � dWs

�:

In particular, V is a viscosity solution on Œ0; T / � .0;1/d of the linear PDE:

0 D �@tV � 1

2Tr�.x ? �/2D2

xV�: (8.41)

For later use, let us denote by

W Qt;x WD W CZ �

t

�.Xt;xs /ds; s 2 Œt; T �;

the Qt;x-Brownian motion defined on Œt; T �.

The Viscosity Supersolution Property By the results of the previous section, wehave OV� is a viscosity supersolution on Œ0; T / � R

dC � Œ0; 1� of the equation


0 � �@t OV� � 1

2Trh�2x2D2

xOV�i

� inf˛2Rd

�˛�Dp

OV� C Trh�x˛Dxp

OV�i

C 1

2j˛j2D2

pOV��: (8.42)

The boundary conditions at p D 0 and p D 1 are immediate:

OV�.�; 1/ D V and OV�.�; 0/ D 0 on Œ0; T � � RdC: (8.43)

We next determine the boundary condition at the terminal time t D T .

Lemma 8.9. For all x 2 RdC and p 2 Œ0; 1�, we have OV�.T; x; p/ � pg.x/.

Proof. Let .tn; xn; pn/n be a sequence in Œ0; T /�RdC �.0; 1/ converging to .T; x; p/

with OV .tn; xn; pn/ �! OV�.T; x; p/, and consider yn WD OV .tn; xn; pn/ C 1=n. Bydefinition of the quantile hedging problem, there is a sequence .�n; ˛n/ 2 OU0 suchthat

1fY tn;xn;yn;�nT �g.Xtn;xnT /�0g � Ptn;pn;˛nT :

This implies that

Ytn;xn;yn;�nT � P

tn;pn;˛nT g.X

tn;xnT /:

Taking the expectation under Qtn;xn , this provides:

yn � EQtn ;xn

�Ytn;xn;yn;�nT

� � EQtn;xn

�Ptn;pn;˛nT g.X

tn;xnT /

�

D E�Ltn;xnT P

tn;pn;˛nT g.X

tn;xnT /

�;

where we denote Ltn;xnT WD exp

� R Ttn �.Xtn;xn

s / � dWs � 12

R Ttn

j�.Xtn;xns /j2ds

.

Then

yn � E�Ptn;pn;˛nT g.x/

�C E�Ptn;pn;˛nT

�Ltn;xnT g.X

tn;xnT / � g.x/��

D png.x/C E�Ptn;pn;˛nT

�Ltn;xnT g.X

tn;xnT / � g.x/��

� png.x/ � E�Ptn;pn;˛nT

ˇˇLtn;xnT g.X

tn;xnT / � g.x/ˇˇ� ; (8.44)

where we used the fact that P tn;pn;˛n is a nonnegative martingale. Now, since thisprocess is also bounded by 1, we have

E�Ptn;pn;˛nT

ˇˇLtn;xnT g.X

tn;xnT /� g.x/

ˇˇ� � E

�ˇˇLtn;xnT g.Xtn;xnT /� g.x/

ˇˇ� �! 0


as n ! 1, by the stability properties of the flow and the dominated conver-gence theorem. Then, by taking limits in (8.44), we obtain that OV�.T; x; p/ Dlimn!1 yn � pg.x/, which is the required inequality. utAn Explicit Minorant of OV The key idea is to introduce the Legendre-Fencheldual of V� with respect to the p�variable in order to remove the non-linearityin (8.42):

v.t; x; q/ WD supp2R

npq � OV�.t; x; p/

o; .t; x; q/ 2 Œ0; T � � .0;1/d � R: (8.45)

By the definition of the function OV , we have

v.:; q/ D 1 for q < 0 and v.:; q/ D supp2Œ0;1�

npq � OV�.:; p/

ofor q > 0: (8.46)

Using the above supersolution property of OV�, we shall prove below that v is anupper-semicontinuous viscosity subsolution on Œ0; T / � .0;1/d � .0;1/ of

� @tv � 1

2Tr��2x2D2

xv� � 1

2j�j2 q2D2

qv � Tr��xDxqv

� � 0 (8.47)

with the boundary condition

v.T; x; q/ � .q � g.x//C : (8.48)

Since the above equation is linear, we deduce from the comparison result an explicitupper bound for v given by the Feynman-Kac representation result:

v.t; x; q/ � Nv.t; x; q/ WD EQt;x

h�Qt;x;qT � g.X

t;xT /�Ci

; (8.49)

on Œ0; T � � .0;1/d � .0;1/, where the processQt;x;q is defined by the dynamics

dQt;x;qs

Qt;x;qs

D �.Xt;xs / � dW Qt;x

s and Qt;x;q.t/ D q 2 .0;1/: (8.50)

Given the explicit representation of Nv, we can now provide a lower bound for theprimal function OV by using (8.46).

We next deduce from (8.49) a lower bound for the quantile hedging problemOV . Recall that the convex envelope OV convp� of OV� with respect to p is given by the

bi-conjugate function

OV convp� .t; x; p/ D supq

˚pq � v.t; x; q/

�


and is the largest convex minorant of OV�. Then, since OV � OV�, it follows from (8.49)that

OV .t; x; p/ � OV�.t; x; p/ � supq

˚pq � Nv.t; x; q/�: (8.51)

Clearly the function Nv is convex in q and there is a unique solution Nq.t; x; p/ to theequation

@[email protected]; x; Nq/ D E

Qt;x

hQt;x;1T 1fQt;x;Nq.T /�g.Xt;xT /g

i

D P

hQt;x; NqT � g.X

t;xT /i

D p; (8.52)

where we have used the fact that dP=dQt;x D Qt;x;1T . Then the maximization on the

right hand side of (8.51) can be solved by the first order condition, and therefore

OV .t; x; p/ � p Nq.t; x; p/ � Nv .t; x; Nq.t; x; p//D Nq.t; x; p/

p � E

Qt;xhQt;x;1T 1f Nq.t;x;p/Qt;x;1

T �g.Xt;xT /gi

CEQt;x

hg.X

t;xT /1f Nq.t;x;p/Qt;x;1

T �g.Xt;xT /gi

D EQt;x

hg.Xt;x

T /1f Nq.t;x;p/Qt;x;1T �g.Xt;xT /g

iDW y.t; x; p/:

The Explicit Solution We finally show that the above explicit minorant y.t; x; p/is equal to OV .t; x; p/. By the martingale representation theorem, there exists acontrol process � 2 U0 such that

Yt;x;y.t;x;p/;�T D g

�Xt;xT

�1f Nq.t;x;p/Qt;x;1

T �g.Xt;xT /g:

Since Ph

Nq.t; x; p/Qt;x;1T � g.X

t;xT /i

D p, by (8.52), this implies that OV .t; x; p/ �y.t; x; p/.

Proof of (8.47) and (8.48). First note that the fact that v is uppersemicontinuous onŒ0; T �� .0;1/d � .0;1/ follows from the lower semicontinuity of OV� and the rep-resentation in the right-hand side of (8.46), which allows to reduce the computationof the sup to the compact set Œ0; 1�. Moreover, the boundary condition (8.48) is animmediate consequence of the right-hand side inequality in (8.43) and (8.46) again.

We now turn to the subsolution property inside the domain. Let ' be a smoothfunction with bounded derivatives and .t0; x0; q0/ 2 Œ0; T /� .0;1/d � .0;1/ be alocal maximizer of v � ' such that .v � '/.t0; x0; q0/ D 0.

(i) We first show that we can reduce to the case where the map q 7! '.�; q/is strictly convex. Indeed, since v is convex, we necessarily haveDqq'.t0; x0; q0/

� 0. Given "; � > 0, we now define '";� by '";�.t; x; q/ WD '.t; x; q/ C "jq �q0j2 C �jq � q0j2.jq � q0j2 C jt � t0j2 C jx � x0j2/. Note that .t0; x0; q0/


is still a local maximizer of U � '";�. Since Dqq'.t0; x0; q0/ � 0, we haveDqq'";�.t0; x0; q0/ � 2" > 0. Since ' has bounded derivatives, we can thenchoose � large enough so that Dqq'";� > 0. We next observe that, if '";�satisfies (8.47) at .t0; x0; q0/ for all " > 0, then (8.47) holds for ' at thispoint too. This is due to the fact that the derivatives up to order two of '";� at.t0; x0; q0/ converge to the corresponding derivatives of ' as " ! 0.

(ii) From now on, we thus assume that the map q 7! '.�; q/ is strictly convex. LetQ' be the Fenchel transform of ' with respect to q, i.e.,

Q'.t; x; p/ WD supq2R

fpq � '.t; x; q/g :

Since ' is strictly convex in q and smooth on its domain, Q' is strictly convex inp and smooth on its domain. Moreover, we have

'.t; x; q/ D supp2R

fpq � Q'.t; x; p/g D J.t; x; q/q � Q'.t; x; J.t; x; q//

on .0; T / � .0;1/d � .0;1/ � int.dom.'//, where q 7! J.�; q/ denotes theinverse of p 7! Dp Q'.�; p/ recall that Q' is strictly convex in p.

We now deduce from the assumption q0 > 0 and (8.46) that we can findp0 2 Œ0; 1� such that v.t0; x0; q0/ D p0q0 � OV�.t0; x0; p0/ which, by using the verydefinition of .t0; x0; p0; q0/ and v, implies that

0 D . OV� � Q'/.t0; x0; p0/ D (local) min. OV� � Q'/ (8.53)

and

'.t0; x0; q0/ D supp2R

fpq0 � Q'.t0; x0; p/g (8.54)

D p0q0 � Q'.t0; x0; p0/ with p0 D J.t0; x0; q0/; (8.55)

where the last equality follows from (8.53) and the strict convexity of the map p 7!pq0 � Q'.t0; x0; p/ in the domain of Q'.

We conclude the proof by discussing three alternative cases depending on thevalue of p0

• If p0 2 .0; 1/, then (8.53) implies that Q' satisfies (8.42) at .t0; x0; p0/ and therequired result follows by exploiting the link between the derivatives of Q' andthe derivatives of its p-Fenchel transform ', which can be deduced from (8.53).

• If p0 D 1, then the first boundary condition in (8.43) and (8.53) imply that.t0; x0/ is a local minimizer of OV�.�; 1/ � Q'.�; 1/ D V � Q'.�; 1/ such that .V �Q'.�; 1//.t0; x0/ D 0. This implies that Q'.�; 1/ satisfies (8.41) at .t0; x0/ so that Q'satisfies (8.42) for ˛ D 0 at .t0; x0; p0/. We can then conclude as in 1. above.


• If p0 D 0, then the second boundary condition in (8.43) and (8.53) imply that.t0; x0/ is a local minimizer of OV�.�; 0/ � Q'.�; 0/ D 0 � Q'.�; 0/ such that 0 �Q'.�; 0/.t0; x0/ D 0. In particular, .t0; x0/ is a local maximum point for Q'.�; 0/so that .@t Q';Dx Q'/.t0; x0; 0/ D 0 and Dxx Q'.t0; x0; 0/ is negative semi-definite.This implies that Q'.�; 0/ satisfies (8.41) at .t0; x0/ so that Q' satisfies (8.42) at.t0; x0; p0/, for ˛ D 0. We can then argue as in the first case. ut

Chapter 9Second Order Stochastic Target Problems

In this chapter, we extend the class of stochastic target problems of the previoussection to the case where the quadratic variation of the control process � is involvedin the optimization problem. This new class of problems is motivated by applicationsin financial mathematics.

We first start by studying in details the so-called problem of hedging undergamma constraints. This simple example already outlines the main difficulties. Byusing the tools from viscosity solutions, we shall first exhibit a minorant for thesuperhedging cost in this context. We then argue by verification to prove that thisminorant is indeed the desired value function.

We then turn to a general formulation of second-order stochastic target problems.Of course, in general, there is no hope to use a verification argument as in theexample of the first section. We therefore provide the main tools in order toshow that the value function is a viscosity solution of the corresponding dynamicprogramming equation.

Finally, Sect. 9.3 provides another application to the problem of superhedgingunder illiquidity cost. We will consider the illiquid financial market introduced byCetin et al. [8], and we will show that our second-order stochastic target frameworkleads to an illiquidity cost which can be characterized by means of a nonlinear PDE.

9.1 Superhedging Under Gamma Constraints

In this section, we focus on an alternative constraint on the portfolio � . Forsimplicity, we consider a financial market with a single risky asset. Let Zt.!/ WDS�1t �t .!/ denote the vector of number of shares of the risky assets held at each

time t and ! 2 �. By definition of the portfolio strategy, the investor has to adjusthis strategy at each time t , by passing the number of shares from Zt to ZtCdt . Hisdemand in risky assets at time t is then given by “dZt”.


123

124 9 Second Order Stochastic Target Problems

In an equilibrium model, the price process of the risky asset would be pushedupward for a large demand of the investor. We therefore study the hedging problemwith constrained portfolio adjustment.

However, market practitioners only focus on the diffusion component of thehedge adjustment dZt , which is given by the so-called gamma, i.e., the Hessian ofthe Black–Scholes prices. The gamma of the hedging strategy is an important riskcontrol parameter indicating the size of the replacement of the hedging portfolioinduced by a stress scenario, i.e., a sudden jump of the underlying asset spot price.A large portfolio gamma leads to two different risks depending on its sign:

– A large positive gamma requires that the seller of the option adjusts his hedgingportfolio by a large purchase of the underlying asset. This is a typical risk thattraders want to avoid because then the price to be paid for this hedging adjustmentis very high and sometimes even impossible because of the limited offer ofunderlying assets on the financial market.

– A negative gamma induces a risk of different nature. Indeed, the hedger has thechoice between two alternative strategies: either adjust the hedge at the expenseof an outrageous market price, or hold the Delta position risk. The latter buy-and-hold strategy does not violate the hedge thanks to the (local) concavity ofthe payoff (negative gamma). There are two ways to understand this result: thesecond-order term in the Taylor expansion has a sign in favor of the hedger, orequivalently, the option price curve is below its tangent which represents the buy-and-hold position.

This problem turns out to present serious mathematical difficulties. The analysisof this section provides a solution of the problem of hedging under upper bound onthe gamma in a very specific situation. The lower bound on the gamma introducesmore difficulties due to the fact that the nonlinearity in the “first guess” equation isnot elliptic.

9.1.1 Problem Formulation

We consider a financial market which consists of one bank account, with constantprice process S0t D 1 for all t 2 Œ0; T �, and one risky asset with price processevolving according to the Black–Scholes model:

Su WD St exp

��12�2.t � u/C �.Wt �Wu/

�; t � u � T:

Here, W is a standard Brownian motion in R defined on a complete probabilityspace .˝;F ; P /. We shall denote by F D fFt , 0 � t � T g the P -augmentation ofthe filtration generated by W .

9.1 Superhedging Under Gamma Constraints 125

Observe that there is no loss of generality in taking S as a martingale, as onecan always reduce the model to this case by judicious change of measure. On theother hand, the subsequent analysis can be easily extended to the case of a varyingvolatility coefficient.

We denote by Z D fZu, t � u � T g the process of number of shares ofrisky asset S held by the agent during the time interval Œt; T �. Then, by the self-financing condition, the wealth process induced by some initial capital y, at time t ,and portfolio strategy Z is given by:

Yu D y CZ u

t

ZrdSr; t � u � T:

In order to introduce constraints on the variations of the hedging portfolio Z, werestrictZ to the class of continuous semimartingales with respect to the filtration F.Since F is the Brownian filtration, we define the controlled portfolio strategyZz;˛;� by

Zz;˛;�u D z C

Z u

t

˛rdr CZ u

t

�r�dWr; t � u � T; (9.1)

where z 2 R is the time t initial portfolio and the control pair .˛; � / are boundedprogressively measurable processes. We denote by Bt the collection of all suchcontrol processes.

Hence, a trading strategy is defined by the triple � WD .z; ˛; � / with z 2 R and.˛; � / 2 Bt . The associated wealth process, denoted by Y y;� , is given by

Y y;�u D y CZ u

t

Z�r dSr; t � u � T; (9.2)

where y is the time t initial capital. We now formulate the gamma constraint in thefollowing way. Let � < 0 < � be two fixed constants. Given some initial capitaly 2 R, we define the set of y-admissible trading strategies by

At .� ; � / WD ˚� D .y; ˛; � / 2 R � Bt W � � ��

�:

As in the previous sections, we consider the super-replication problem of someEuropean-type contingent claim g.ST /

v.t; St / WD inf˚y W Y y;�T � g.ST / a.s. for some � 2 At .� ; � /

�: (9.3)

Remark 9.1. The above set of admissible strategies seems to be very restrictive.We will see later that one can possibly enlarge, but not so much! The fundamentalreason behind this can be understood from the following result due to Bank andBaum [1] and restated here in the case of the Brownian motion:


Lemma 9.2. Let � be a progressively measurable process withR 10

j�t j2dt < 1,P�a.s. Then for all " > 0, there is a process �" with d�"t D ˛"t dt for some

progressively measurable ˛" withR 10 j˛"t jdt < 1, P�a.s. such that

supt�1

��Z 1

0

�tdWt �Z 1

0

�"t dWt

��L1

� ":

Given this result, it is clear that without any constraint on the process ˛ in thestrategy �, the superhedging cost would be obviously equal to the Black–Scholesunconstrained price. Indeed, the previous lemma says that one can approximatethe Black–Scholes hedging strategy by a sequence of hedging strategies with zerogamma without affecting the liquidation value of the hedging portfolio by morethat ".

9.1.2 Hedging Under Upper Gamma Constraint

In this section, we consider the case

� D �1 and we denote At .� / WD At .�1; � /:

Our goal is to derive the following explicit solution: v.t; St / is the (unconstrained)Black–Scholes price of some convenient face-lifted contingent claim Og.ST /, wherethe function Og is defined by

Og.s/ WD hconc.s/C � s ln s with h.s/ WD g.s/ � � s ln s;

and hconc denotes the concave envelope of h. Observe that this function can becomputed easily. The reason for introducing this function is the following:

Lemma 9.3. Og is the smallest function satisfying the conditions

.i/ Og � g ; and .ii/ s 7�! Og.s/ � � s ln s is concave.

The proof of this easy result is omitted.

Theorem 9.4. Let g be a lower-semicontinuous mapping on RC with

s 7�! Og.s/ � C s ln s convex for some constant C: (9.4)

Then the value function (9.3) is given by

v.t; s/ D Et;s Œ Og .ST /� for all .t; s/ 2 Œ0; T / � .0;1/:


Discussion 1. We first make some comments on the model. Intuitively, we expectthe optimal hedging portfolio to satisfy

OZu D vs.u; Su/;

where v is the minimal superhedging cost. Assuming enough regularity, it followsfrom Ito’s formula that

d OZu D Audu C �Suvss.u; Su/dWu;

whereA.u/ is given in terms of derivatives of v. Compare this equation with (9.1)to conclude that the associated gamma is

O�u D Su vss.u; Su/:

Therefore, the bound on the process O� translates to a bound on svss . Notice that,by changing the definition of the process � in (9.1), we may bound vss insteadof svss . However, we choose to study svss because it is a dimensionless quantity,i.e., if all the parameters in the problem are increased by the same factor, svssstill remains unchanged.

2. Intuitively, we expect to obtain a similar type solution to the case of portfolioconstraints. If the Black–Scholes solution happens to satisfy the gamma con-straint, then it solves the problem with gamma constraint. In this case v satisfiesthe PDE �@t � � L� D 0. Since the Black–Scholes solution does not satisfy thegamma constraint, in general, we expect that the function v solves the variationalinequality

min˚�@t � � L�; � � s�ss

� D 0: (9.5)

3. An important feature of the log-normal Black and Scholes model is that thevariational inequality (9.5) reduces to the Black–Scholes PDE �@t��L� D 0 aslong as the terminal condition satisfies the gamma constraint (in a weak sense).From Lemma 9.3, the face-lifted payoff function Og is precisely the minimalfunction above g which satisfies the gamma constraint (in a weak sense). Thisexplains the nature of the solution reported in Theorem 9.4, namely, v.t; St / isthe Black–Scholes price of the contingent claim Og .ST /.

Dynamic Programming and Viscosity Property. We now turn to the proof ofTheorem 9.4. We shall denote

Ov.t; s/ WD Et;s Œ Og .ST /� :

It is easy to check that Ov is a smooth function satisfying

@t C LOv D 0 and s Ovss � � on Œ0; T / � .0;1/ : (9.6)


1. We start with the inequality v � Ov. For t � u � T , we set

z WD Ovs.t; s/; ˛u WD .@t C L/Ovs.u; Su/; �u WD Su Ovss.u; Su/;

and we claim that

.˛; � / 2 Bt and � � � ; (9.7)

so that the corresponding control � D .y; ˛; � / 2 At .� /. Before verifying thisclaim, let us complete the proof of the required inequality. Since g � Og, we have

g .ST / � Og .ST / D Ov .T; ST /

D Ov.t; St /CZ T

t

.@t C L/Ov.u; Su/du C Ovs.u; Su/dSu

D Ov.t; St /CZ T

t

Z�u dSuI

in the last step, we applied Ito’s formula to Ovs . Now, observe that the right handside of the previous inequality is the liquidation value of the portfolio started fromthe initial capital Ov.t; St / and using the portfolio strategy �. By the definition ofthe superhedging problem (9.3), we conclude that v � Ov.

It remains to prove (9.7). The upper bound on � follows from (9.6). As forthe lower bound, it is obtained as a direct consequence of condition (9.4). Usingagain (9.6) and the smoothness of Ov, we see that 0D f.@t CL/Ovgs D .@t CL/OvsC�2s Ovss , so that ˛ D ��2� is also bounded.

2. The proof of the reverse inequality v � Ov requires much more effort. The mainstep is the following (half) dynamic programming principle.

Lemma 9.5. Let y 2 R, � 2 At .� / be such that Y y;�T � g .ST / P�a.s. Then

Yy;�

� v .; S / ; P � a.s.

for all stopping times valued in Œt; T �.

The proof of this claim is easy and follows from the same argument than thecorresponding one in the standard stochastic target problems of the previous chapter.

We continue by stating two lemmas whose proofs rely heavily on the abovedynamic programming principle and will be reported later. We denote as usual byv� the lower-semicontinuous envelope of v.

Lemma 9.6. The function v� is a viscosity supersolution of the equation

�.@t C L/v� � 0 on Œ0; T / � .0;1/:

Lemma 9.7. The function s 7�! v�.t; s/� � s ln s is concave for all t 2 Œ0; T �.Before proceeding to the proof of these results, let us show how the remaining

inequality v � Ov follows from it. Given a trading strategy in At .� /, the associated


wealth process is a square integrable martingale, and therefore a supermartingale.From this, one easily proves that v�.T; s/ � g.s/. By Lemma 9.7, v�.T; �/ alsosatisfies requirement (ii) of Lemma 9.3, and therefore

v�.T; �/ � Og:In view of Lemma 9.6, v� is a viscosity supersolution of the equation �Lv� D 0 andv�.T; �/D Og. Since Ov is a viscosity solution of the same equation, it follows from theclassical comparison theorem that v� � Ov.

Hence, in order to complete the proof of Theorem 9.4, it remains to proveLemmas 9.6 and 9.7.

Proof of Lemmas 9.6 and 9.7. We split the argument in several steps:

1. We first show that the problem can be reduced to the case where the controls.˛; � / are uniformly bounded. For " 2 .0; 1�, set

A"

t WD ˚� D .y; ˛; � / 2 At .� / W j˛.:/j C j� .:/j � "�1� ;

and

v".t; St / D infny W Y y;�T � g.ST / P � a.s. for some � 2 A"

t

o:

Let v"� be the lower-semicontinuous envelope of v". It is clear that v" also satisfiesthe dynamic programming equation of Lemma 9.5.

Since

v�.t; s/ D lim inf� v".t; s/ D lim inf"!0;.t 0;s0/!.t;s/

v"�.t 0; s0/ ;

we shall prove that

� .@t C L/v" � 0 in the viscosity sense; (9.8)

and the statement of the lemma follows from the classical stability result ofTheorem 6.8.

2. We now derive the implications of the dynamic programming principle ofLemma 9.5 applied to v". Let ' 2 C1.R2/ and .t0; s0/ 2 .0; T /� .0;1/ satisfy

0 D .v"� � '/.t0; s0/ D min.0;T /�.0;1/

.v"� � '/ I

in particular, we have v"� � '. Choose a sequence .tn; sn/ ! .t0; s0/ so thatv".tn; sn/ converges to v"�.t0; s0/. For each n, by the definition of v" and thedynamic programming, there are yn 2 Œv".tn; sn/; v".tn; sn/ C 1=n�, hedgingstrategies �n D .zn; ˛n; �n/ 2 A"

tnsatisfying

Yyn;�nn

� v" .n; Sn/ � 0


for every stopping time n valued in Œtn; T �. Since v" � v"� � ',

yn CZ n

tn

Z�nu dSu � ' .n; Sn/ � 0:

Observe that

ˇn WD yn � '.tn; sn/ �! 0 as n �! 1:

By Ito’s formula, this provides

Mnn

� Dnn

C ˇn; (9.9)

where

Mnt WD

Z t

0

�'s.tn C u; StnCu/ � Y �ntnCu

�dStnCu

Dnt WD �

Z t

0

.@t C L/'.tn C u; StnCu/du :

We now choose conveniently the stopping time n. For all h > 0, define thestopping time

n WD .tn C h/ ^ inf fu > tn W jln .Su=sn/j � 1g :

3. By the smoothness of L', the integrand in the definition ofMn is bounded up tothe stopping time n and therefore taking expectation in (9.9) provides

�Etn;sn"Z t^n

0

.@t C L/'.tn C u; StnCu/du

#� �ˇn

We now send n to infinity, divide by h, and take the limit as h & 0. The requiredresult follows by dominated convergence.

4. It remains to prove Lemma 9.7. The key-point is the following result, which is aconsequence of Theorem 5.5.

Lemma 9.8. Let .˚anu ; u � 0g�

nand .

˚bnu ; u � 0g�

nbe two sequences of real-

valued, progressively measurable processes that are uniformly bounded in n. Let.tn; sn/ be a sequence in Œ0; T /�.0;1/ converging to .0; s/ for some s > 0. Supposethat

Mnt^n WD

Z tnCt^n

tn

��n C

Z u

tn

anr dr CZ u

tn

bnr dSr

�dSu

� ˇn C C t ^ n


for some real numbers .�n/n, .ˇn/n, and stopping times .n/n � tn. Assume furtherthat, as n tends to infinity,

ˇn �! 0 and t ^ n �! t ^ 0 P � a:s:;

where 0 is a strictly positive stopping time. Then

(i) limn!1 �n D 0.(ii) limu!0 essinf0�r�u bu � 0, where b be a weak limit process of .bn/n.

5. We now complete the proof of Lemma 9.7. We start exactly as in the previousproof by reducing the problem to the case of uniformly bounded controls, andwriting the dynamic programming principle on the value function v".

By a further application of Ito’s lemma, we see that

Mn.t/ DZ t

0

��n C

Z u

0

anr dr CZ u

0

bnr dStnCr�

dStnCu (9.10)

where

�n WD 's.tn; sn/ � zn;

an.r/ WD .@t C L/'s.tn C r; StnCr / � ˛ntnCr l;

bnr WD 'ss.tn C r; StnCr /� � ntnCrStnCr

:

Observe that the processes an:^n and bn:^n are bounded uniformly in n since .@t CL/'s and 'ss are smooth functions. Also since .@tCL/' is bounded on the stochasticinterval Œtn; n�, it follows from (9.9) that

Mnn

� C t ^ n C ˇn

for some positive constant C . We now apply the results of Lemma 9.8 to themartingalesMn. The result is

limn!1 zn D 's.t0; y0/ and lim

t!0ess inf

0�u�t bt � 0;

where b is a weak limit of the sequence .bn/. Recalling that � n.t/ � � , thisprovides that

�s0'ss.t0; s0/C � � 0:

Hence, v"� is a viscosity supersolution of the equation �s.v�/ss C � � 0, and therequired result follows by the stability result of Theorem 6.8. ut


9.1.3 Including the Lower Bound on the Gamma

We now turn to our original problem (9.3) of superhedging under upper and lowerbounds on the gamma process.

Following the same intuition as in point 2 of the discussion subsequent toTheorem 9.4, we guess that the value function v should be characterized by thePDE:

F.s; @tu; uss/ WD min˚�.@t C L/u; � � suss; suss � �

� D 0;

where the first item where the first item of the minimum expression says that thevalue function should be dominating the Black–Scholes solution, and the two nextones inforce the constraint on the second derivative.

This first guess equation is however not elliptic because the third item of theminimum is increasing in uss . This would divert us from the world of viscositysolutions and the maximum principle. But of course, this is just a guess, andwe should expect, as usual in stochastic control, to obtain an elliptic dynamicprogramming equation.

To understand what is happening in the present situation, we have to go backto the derivation of the DPE from dynamic programming principle in the previoussubsection. In particular, we recall that in the proof of Lemmas 9.6 and 9.7, wearrived at the inequality (9.9):

Mnn

� Dnn

C ˇn;

where

Dnt WD �

Z t

0

.@t C L/'.tn C u; StnCu/du;

andMn is given by (9.10), after an additional application of Ito’s formula,

Mn.t/ DZ t

0

��n C

Z u

0

anr dr CZ u

0

bnr dStnCr�

dStnCu ;

with

�n WD 's.tn; sn/� zn;

an.r/ WD .@t C L/'s.tn C r; StnCr /� ˛ntnCr ;

bnr WD 'ss.tn C r; StnCr / � � ntnCrStnCr

:

To gain more intuition, let us suppress the sequence index n, set ˇn D 0, and takethe processes a and b to be constant. Then, we are reduced to the process


M.t/ D �.St0Ct � St0/C a

Z t0Ct

t0

.u � t0/dSu C b

2

�.St0Ct � St0/

2 �Z t

t0

�2S2u du

�:

This decomposition reveals many observations:

• The second term should play no role as it is negligible in small time compared tothe other ones.

• The requirement M.:/ � D.:/ implies that b � 0 because otherwise the thirdterm would dominate the other two ones, by the law of iterated logarithm ofthe Brownian motion, and would converge to C1 violating the upper boundD.Since b � 0 and � � � � � , this provides

� � s'ss � sbt0 � � :

• We next observe that, by taking the liminf of the third term, the squared difference.St0Ct � St0/

2 vanishes. So we may continue as in Step 3 of the proof ofLemmas 9.6 and 9.7, taking expected values, normalizing by h, and sending hto zero. Because of the finite variation component of the third term

R tt0�2S2u du,

this leads to

0 � �@t' � 1

2�2s2'ss � bt0

2�2s2

D �@t' � 1

2�2s2.'ss C bt0/:

Collecting the previous inequalities, we arrive at the supersolution property:

OF .s; @t '; 'ss/ � 0;

where

OF .s; @t '; 'ss/ D supˇ�0

F.s; @t '; 'ss C ˇ/:

A remarkable feature of the nonlinearity OF is that it is elliptic! In fact, it is easy toshow that OF is the smallest elliptic majorant of F . For this reason, we call OF theelliptic majorant of F .

The above discussion says all about the derivation of the supersolution property.However, more conditions on the set of admissible strategies need to be imposed inorder to turn it into a rigorous argument. Once the supersolution property is proved,one also needs to verify that the subsolution property holds true. This also requiresto be very careful about the set of admissible strategies. Instead of continuing thisexample, we shall state without proof the viscosity property, without specifying theprecise set of admissible strategies. This question will be studied in details in thesubsequent paragraph, where we analzse a general class of second-order stochastictarget problems.


Theorem 9.9. Under a convenient specification of the set A.� ; � /, the valuefunction v is a viscosity solution of the equation

OF .s; @t v; vss/ D 0 on Œ0; T / � RC:

9.2 Second Order Target Problem

In this section, we introduce the class of second-order stochastic target problemsmotivated by the hedging problem under gamma constraints of the previous section.


The finite time horizon T 2 .0;1/ will be fixed throughout this section. As usual,fWtgt2Œ0;T � denotes a d -dimensional Brownian motion on a complete probabilityspace .�;F ; P / and F D .Ft /t2Œ0;T � the corresponding augmented filtration.

State Processes. We first start from the uncontrolled state processX defined by thestochastic differential equation

Xt D x CZ t

s

�.Xu/du CZ t

s

�.Xu/dWu; t 2 Œs; T �:

Here, � and � are assumed to satisfy the usual Lipschitz and linear growthconditions so as to ensure the existence of a unique solution to the above SDE.We also assume that �.x/ is invertible for all x 2 R

d .The control is defined by the R

d -valued process fZt gt2Œs;T � of the form

Zt D z CZ t

s

Ardr CZ t

s

�rdXs;xr ; t 2 Œs; T �; (9.11)

�t D CZ t

s

ardr CZ t

s

�rdXs;xr ; t 2 Œs; T �; (9.12)

where f�tgt2Œs;T � takes values in Sd . Notice that both Z and � have continuoussample paths, a.s.

Before specifying the exact class of admissible control processesZ, we introducethe controlled state process Y defined by

dYt D f .t; Xs;xt ; Yt ; Zt ; �t / dt CZt ı dXs;x

t ; t 2 Œs; T /; (9.13)

9.2 Second Order Target Problem 135

with initial data Ys D y. Here, ı denotes the Fisk–Stratonovich integral. Due tothe form of the Z process, this integral can be expressed in terms of standard Itointegral:

Zt ı dXs;xt D Zt � dXs;x

t C 1

2TrŒ�T��t �dt:

The function f W Œ0; T /�Rd �R�R

d �Sd ! R, appearing in the controlled stateequation (9.13), is assumed to satisfy the following Lipschitz and growth conditions:

(A1) For all N > 0, there exists a constant FN such that

jf .t; x; y; z; / � f .t; x; y0; z; /j � FN jy � y0j

for all .t; x; y; z; / 2 Œ0; T � � Rd � R � R

d � Sd , y0 2 R satisfying

maxfjxj; jyj; jy0j; jzj; j jg � N:

(A2) There exist constants F and p � 0 such that

jf .t; x; y; z; /j � F.1C jxjp C jyj C jzjp C j jp/

for all .t; x; y; z; / 2 Œ0; T � � Rd � R � R

d � Sd .(A3) There exists a constant c0 > 0 such that

f .t; x; y0; z; / � f .t; x; y; z; / � �c0.y0 � y/ for every y0 � y;

and .t; x; z; / 2 Œ0; T / � Rd � R

d � Sd .

Admissible Control Processes As outlined in Remark 9.1, the control processesmust be chosen so as to exclude the possibility of avoiding the impact of the gammaprocess by approximation.

We shall fix two constantsB; b � 0 throughout, and we refrain from indexing allthe subsequent classes of processes by these constants. For .s; x/ 2 Œ0; T ��R

d , wedefine the norm of an F�progressively measurable process fHt gt2Œs;T � by,

kHkB;bs;x WD�� sups�t�T

jHt j1C jXs;x

t jB��Lb

:

For all m > 0, we denote by As;xm;b be the class of all (control) processes Z of the

form (9.11), where the processesA; a; � are F-progressively measurable and satisfy:

kZkB;1s;x � m; k� kB;1s;x � m; k�kB;2s;x � m; (9.14)

kAkB;bs;x � m; kakB;bs;x � m: (9.15)


The set of admissible portfolio strategies is defined by

As;x WD[

b2.0;1�

[m�0

As;xm;b : (9.16)

The Stochastic Target Problem. Let g W Rd ! R be a continuous function

satisfying the linear growth condition,

(A4) g is continuous, and there exist constants G and p such that

jg.x/j � G.1C jxjp/ for all x 2 Rd :

For .s; x/ 2 Œ0; T � � Rd , we define:

V.s; x/ WD infny 2 R W Y s;x;y;ZT � g.X

s;xT /;P � a.s. for some Z 2 As;x

o:

(9.17)

9.2.2 The Geometric Dynamic Programming

As usual, the key ingredient in order to obtain a PDE satisfied by our value functionV is the derivation of a convenient dynamic programming principle obtained byallowing the time origin to move. In the present context, we have the followingstatement which is similar to the case of standard stochastic target problems.

Theorem 9.10. For any .s; x/ 2 Œ0; T / � Rd , and a stopping time 2 Œs; T �,

V.s; x/ D inf˚y 2 R W Y s;x;y;Z � V

;Xs;x

�; P � a.s. for some Z 2 As;x

�:

(9.18)

The proof of this result can be consulted in [37]. Because the processesZ and �are not allowed to jump, the proof is more involved than in the standard stochastictarget case, and uses crucially the nature of the above defined class of admissiblestrategies As;x .

To derive the dynamic programming equation, we will split the geometricdynamic programming principle in the following two claims:

(GDP1) For all " > 0, there exist y" 2 ŒV .s; x/; V .s; x/ C "� and Z" 2 As;x s.t.

Ys;x;y";Z" � V

;X

s;x

�; P � a.s. (9.19)

(GDP2) For all y < V.s; x/ and every Z 2 As;x ,

P

hYs;x;y;Z

� V;X

s;x

�i< 1: (9.20)


Notice that (9.18) is equivalent to (GDP1) and (GDP2). We shall prove that(GDP1) and (GDP2) imply that the value function V is a viscosity supersolu-tion and subsolution, respectively, of the corresponding dynamic programmingequation.


Similar to the problem of hedging under gamma constraints, the dynamic program-ming equation corresponding to our second-order target problem is obtained as theparabolic envelope of the first guess equation:

� @tv C Of :; v;Dv;D2v

� D 0 on Œ0; T / � Rd ; (9.21)

where

Of .t; x; y; z; / WD supˇ2Sd

C

f .t; x; y; z; C ˇ/ (9.22)

is the smallest majorant of f which is nonincreasing in the argument and is calledthe parabolic envelope of f . In the following result, we denote by V � and V� theupper- and lower-semicontinuous envelopes of V :



V .t 0; x0/

for .t; x/ 2 Œ0; T � � Rd .

Theorem 9.11. Assume that V is locally bounded, and let conditions (A1–A4) holdtrue. Then V is a viscosity solution of the dynamic programming equation (9.21)on Œ0; T � � R

d , i.e., V� and V � are, respectively, viscosity super-solution andsubsolution of (9.21).

Proof of the Viscosity Subsolution Property. Let .t0; x0/ 2 Q and ' 2 C1 .Q/be such that

0 D .V � � '/.t0; x0/ > .V� � '/.t; x/ for Q 3 .t; x/ ¤ .t0; x0/: (9.23)

In order to show that V � is a subsolution of (9.21), we assume to the contrary, i.e.,suppose that there is ˇ 2 SdC satisfying

� @'

@t.t0; x0/C f

t0; x0; '.t0; x0/;D'.t0; x0/;D

2'.t0; x0/C ˇ�> 0: (9.24)

We will then prove the sub-solution property by contradicting (GDP2).


(i) Set

.t; x/ WD '.t; x/C ˇ.x � x0/ � .x � x0/;

h.t; x/ WD �@ @t.t; x/C f

t; x; .t; x/;D .t; x/;D2 .t; x/

�:

In view of (9.24), h.t0; x0/ > 0. Since the nonlinearity f is continuous and 'is smooth, the subset

N WD f.t; x/ 2 Q \ B1.t0; x0/ W h.t; x/ > 0gis an open bounded neighborhood of .t0; x0/. Here, B1.t0; x0/ is the unit ballof Q centered at .t0; x0/. Since .t0; x0/ is defined by (9.23) as the point of strictmaximum of the difference .V � � '/, we conclude that

� � WD max@N

.V � � '/ < 0: (9.25)

Next, we fix � 2 .0; 1/, and chooseOt ; Ox�

so that

Ot ; Ox� 2 N ; j Ox � x0j � ��; andˇV

Ot ; Ox� � ' Ot ; Ox�ˇ � ��: (9.26)

Set OX WD X Ot ; Ox and define a stopping time by

WD infnt � Ot W .t; OXt/ 62 N

o:

Then, > Ot . The path-wise continuity of OX implies that .; OX/ 2 @N . Then,by (9.25),

V �.; OX/ � '.; OX/� �: (9.27)

(ii) Consider the control process

Oz WD D Ot ; Ox�

; OAt WD LD .t; OXt /1ŒOt ;/.t/ and O�t WD D2 .t; OXt/1ŒOt ;/.t/

so that, for t 2 �Ot ; �,

OZt WD Oz CZ t

OtOArdr C

Z t

OtO�rd OXr D D .t; OXt/:

Since N is bounded and ' is smooth, we directly conclude that OZ 2 AOt ; Ox .

(iii) Set Oy < V.Ot ; Ox/, OYt WD YOt ; Ox; Oy; OZt and O�t WD .t; OXt /. Clearly, the process �

is bounded on ŒOt ; �. For later use, we need to show that the process OY is alsobounded. By definition, OYOt < �Ot . Consider the stopping times

0 WD infnt � Ot W �t D OYt

o;


and, with N WD ��1,

� WD infnt � Ot W OYt D �t �N

o:

We will show that for a sufficiently large N , both 0 D � D . This provesthat as � , OY is also bounded on ŒOt ; �.

Set O WD ^0^�. Since both processes OY and� solve the same stochastic

differential equation, it follows from the definition of N that for t 2 ŒOt ; O�

d�t � OYt

�D

�@

@t.t; OXt/� f

t; OXt ; OYt ; OZt ; O�t

� dt

�hf

t; OXt; �t ; OZt ; O�t

�� f


�idt

� FN

�t � OYt

�dt ;

by the local Lipschitz property (A1) of f . Then

0 � � O � OY O ��Ot � OYOt

�eFN T � 1

2kˇk�2eFN T �2; (9.28)

where the last inequality follows from (9.26). This shows that, for � sufficientlysmall, O < �, and therefore the difference � � OY is bounded. Since � isbounded, this implies that OY is also bounded for small �.

(iv) In this step we will show that for any initial data

Oy 2 ŒV .Ot ; Ox/ � ��; V .Ot ; Ox//;

we have OY � V.;X/. This inequality is in contradiction with (GDP2) asOYOt D Oy < V.Ot ; Ox/. This contradiction proves the subsolution property.

Indeed, using Oy � V.Ot ; Ox/ � �� and V � V � � ' together with (9.25)and (9.26), we obtain the following sequence of inequalities:

OY � V.; OX/ � OY � '.; OX/C �

D Œ Oy � '.Ot ; Ox/C ��CZ

Ot

hd OYt � d'.t; OXt/

i

� �.1 � 2�/CZ

Ot

hf


�dt

C OZt ı d OXt � d'.t; OXt/i

� �.1 � 2�/C 1

2ˇ

OX � Ox�

� OX � Ox

�


CZ

Ot

hf

t; OXt; OYt ; OZt ; O�t

�dt C OZt ı d OXt � d .t; OXt/

i

� �.1 � 2�/CZ

Ot

hf


�dt

C OZt ı d OXt � d .t; OXt/i;

where the last inequality follows from the nonnegativity of the symmetricmatrix ˇ. We next use Ito’s formula and the definition of N to arrive at

OY�V.; OX/ � �.1�2�/CZ

Ot

hf .t; OXt ; OYt ; OZt ; O�t /�f .t; OXt ; �t ; OZt ; O�t /

idt:

In the previous step, we prove that OY and � are bounded, say by N . Since thenonlinearity f is locally bounded, we use the estimate (9.28) to conclude that

OY � V; OX

�� .1� 2�/� 1

2kˇkTFN eFNT �2�2 � 0

for all sufficiently small �. This is in contradiction with (GDP2). Hence, theproof of the viscosity subsolution property is complete.

Proof of the Viscosity Supersolution Property. We first approximate the valuefunction by

V m.s; x/ WD inffy 2 R j 9Z 2 As;xm so that Y s;x;y;ZT � g.X

s;xT /; a:s:g:

Then, similar to (9.20), we can prove the following analogue statement of (GDP1)for V m:

(GDP1m) For every " > 0 and stopping time 2 Œs; T �, there exist Z" 2 As;xm

and y" 2 ŒV m.s; x/; V m.s; x/C "� such that Y s;x;y";Z" � V m;X

s;x

�.

Lemma 9.12. V m� is a viscosity supersolution of (9.21). Consequently, V� is aviscosity supersolution of (9.21).

Proof. Choose .t0; x0/ 2 Œs; T / � Rd and ' 2 C1.Œs; T / � R

d / such that

0 D .V m�;s � '/.t0; x0/ D min.t;x/2Œs;T /�Rd

.V m�;s � '/.t; x/ :

Let .tn; xn/n�1 be a sequence in Œs; T / � Rd such that .tn; xn/ ! .t0; x0/ and

V m.tn; xn/ ! V m�;s .t0; x0/. There exist positive numbers "n ! 0 such that foryn D V m.tn; xn/C "n, there exists Zn 2 Atn;xn

m with

Y nT � g.XnT /;


where we use the compact notation .Xn; Y n/ D .Xtn;xn ; Y tn;xn;yn;Zn/ and

Znr D zn C

Z r

tn

Anudu CZ r

tn

� nu dXn

u ;

� nr D n C

Z r

tn

anu du CZ r

tn

�nu dXnu ; r 2 Œtn; T �:

Moreover, jznj; j nj � m.1 C jxnjp/ by assumption (9.14). Hence, by passing toa subsequence, we can assume that zn ! z0 2 R

d and n ! 0 2 Sd . Observethat ˛n WD yn � '.tn; xn/ ! 0. We choose a decreasing sequence of numbersın 2 .0; T � tn/ such that ın ! 0 and ˛n=ın ! 0. By (GDP1m),

Y ntnCın � V mtn C ın; X

ntnCın

�;

and therefore

Y ntnCın � yn C ˛n � 'tn C ın; X

ntnCın

� � '.tn; xn/;

which, after two applications of Ito’s formula, becomes

˛n CZ tnCın

tn

f .r;Xn

r ; Ynr ; Z

nr ; �

nr /� 't.r; X

nr /

�dr

C zn �D'.tn; xn/

� � XntnCın � xn

�

CZ tnCın

tn

�Z r

tn

ŒAnu � LD'.u; Xnu /�du

�0ı dXn

r

CZ tnCın

tn

�Z r

tn

Œ� nu �D2'.u; Xn

u /�dXnu

�0ı dXn

r � 0: (9.29)

It is shown in Lemma 9.13 that the sequence of random vectors

0BBBBB@

ı�1n

R tnCıntn

Œf .r; Xnr ; Y

nr ; Z

nr ; �

nr /� 't.r; X

nr /�dr

ı�1=2n ŒXn

tnCın � xn�ı�1n

R tnCıntn

R rtnŒAnu � LD'.u; Xn

u /�du�0 ı dXn

r

ı�1n

R tnCıntn

R rtnŒ� n

u �D2'.u; Xnu /�dX

nu

�0 ı dXnr

1CCCCCA; n � 1 ; (9.30)

converges in distribution to

0BBBB@

f .t0; x0; '.t0; x0/; z0; 0/ � 't .t0; x0/�.x0/W1

01

2W1 � �.x0/T

0 �D2'.t0; x0/

��.x0/W1

1CCCCA : (9.31)


Set �n D jzn �D'.tn; xn/j, and assume ı�1=2n �n ! 1 along a subsequence. Then,

along a further subsequence, ��1n .zn �D'.tn; xn// converges to some �0 2 R

d with

j�0j D 1: (9.32)

Multiplying inequality (9.29) with ı�1=2n ��1

n and passing to the limit yields

�0 � �.x0/W1 � 0;

which, since �.x0/ is invertible, contradicts (9.32). Hence, the sequence .ı�1=2n �n/

has to be bounded, and therefore, possibly after passing to a subsequence,

ı�1=2n Œzn �D'.tn; xn/� converges to some �0 2 R

d :

It follows that z0 D D'.t0; x0/. Moreover, we can divide inequality (9.29) by ınand pass to the limit to get

f .t0; x0; '.t0; x0/;D'.t0; x0/; 0/� 't.t0; x0/

C �0 � �.x0/W1 C 1

2W1 � �.x0/TŒ 0 �D2'.t0; x0/��.x0/W1 � 0: (9.33)

Since the support of the random vectorW1 is Rd , it follows from (9.33) that

f .t0; x0; '.t0; x0/;D'.t0; x0/; 0/� 't .t0; x0/

C�0 � �.x0/w C 1

2w � �.x0/TŒ 0 �D2'.t0; x0/��.x0/w � 0;

for all w 2 Rd . This shows that

f .t0; x0; '.t0; x0/;D'.t0; x0/; 0/� 't .t0; x0/ � 0 and ˇ WD 0 �D2'.t0; x0/ � 0;

and therefore,

�'t .t0; x0/C supˇ2Sd

C

f .t0; x0; '.t0; x0/;D'.t0; x0/;D2'.t0; x0/C ˇ/ � 0 :

This proves that V m is a viscosity supersolution.Since by definition,

V D infmV m;

by the classical stability property of viscosity solutions, V� is also a viscositysupersolution of the DPE (9.21). In fact, this passage to the limit does not fall exactlyinto the stability result of Theorem 6.8, but its justification follows the lines of theproof of stability, the interested reader can find the detailed argument in Corollary5.5 in [12]. utLemma 9.13. The sequence of random vectors (9.30), on a subsequence, convergesin distribution to (9.31).


Proof. Define a stopping time by

n WD inffr � tn W Xnr … B1.x0/g ^ .tn C ın/ ;

where B1.x0/ denotes the open unit ball in Rd around x0. It follows from the fact

that xn ! x0 that

P Œn < tn C ın� ! 0

so that in (9.30), we may replace the upper limits of the integrations by n insteadof tn C ın.

Therefore, in the interval Œtn; n�, the process Xn is bounded. Moreover, in viewof (9.15) so are Zn, � n, and �n.

Step 1. The convergence of the second component of (9.30) is straightforward andthe details are exactly as in Lemma 4.4 [13].

Step 2. Let B be as in (9.14). To analyze the other components, set

An;� WD supu2Œtn;T �

jAnuj1C jXn

u jB ;

so that, by (9.15),

kAn;�kL.1=m/.˝;P/ � m: (9.34)

Moreover, since on the interval Œtn; n�, Xn is uniformly bounded by a deterministicconstant C.x0/ depending only on x0,

ˇAnu

ˇ � C.x0/ An;� � C.x0/m; 8 u 2 Œtn; n�:

(Here and below, the constant C.x0/ may change from line to line.) We define an;�similarly. Then, it also satisfies the above bounds as well. In view of (9.15), also an;�satisfies (9.34). Moreover, using (9.14), we conclude that �nu is uniformly boundedby m.

Step 3. Recall that d� nu D anu du C �nu dXn

u , � ntn

D n. Using the notation and theestimates of the previous step, we directly calculate that

supt2Œtn;n�

j� nt � nj � C.x0/ına

n;� CˇˇZ n

tn

�nu � �ndu

ˇˇ C

ˇˇZ n

tn

�nu �.Xnu /dWu

ˇˇ

WD I n1 C I n2 C I n3 :

Then,

EŒ.I n3 /2� � E

�Z n

tn

j�nu j2j� j2du

�� ın m

2C.x0/2:


Hence, I n3 converges to zero in L2. Therefore, it also converges almost surely on asubsequence. We prove the convergence of I n2 using similar estimates. Since an;�satisfies (9.34),

EŒ.I n1 /.1=m/� � .C.x0/ın/

.1=m/ EŒjan;�j.1=m/� � .C.x0/ın/.1=m/ m:

Therefore, I n1 converges to zero in L.1=m/ and consequently on almost surely on asubsequence.

Hence, on a subsequence, � nt is uniformly continuous. This together with

standard techniques used in Lemma 4.4 of [13] proves the convergence of the firstcomponent of (9.30).

Step 4. By integration by parts,

Z n

tn

Z t

tn

AnududXnt D .Xn

n� Xn

tn/

Z n

tn

Anudu �Z n

tn

.Xnu �Xn

tn/Anudu:

Therefore,

ˇˇ 1ın

Z n

tn

Z t

tn

AnududXnt

ˇˇ � C.x0/ sup

t2Œtn;n�jXn

t � Xntn

j An;�:

Also Xn is uniformly continuous and An;� satisfies (9.34). Hence, we can show thatthe above terms, on a subsequence, almost surely converge to zero. This implies theconvergence of the third term.Step 5. To prove the convergence of the final term, it suffices to show that

J n WD 1

ın

Z n

tn

Z t

tn

Œ� nu � n�dXn

u ı dXnt

converges to zero. Indeed, since n ! 0, this convergence together with thestandard arguments of Lemma 4.4 of [13] yields the convergence of the fourthcomponent.

Since on Œtn; n� Xn is bounded, on this interval j�.Xnt /j � C.x/. Using this

bound, we calculate that

EŒ.J n/2� � C.x0/4

ı2n

Z tnCın

tn

Z t

tn

E�1Œtn;n� j� n

u � nj2�

du dt

� C.x0/4E

"sup

t2Œtn;n�j� n

u � nj2#

DW C.x0/4E�.en/2

�:

In Step 3, we proved the almost sure convergence of en to zero. Moreover, by (9.14),jenj � m. Therefore, by dominated convergence, we conclude that J n converges tozero in L2. Thus almost everywhere on a subsequence. ut

9.3 Superhedging Under Illiquidity Cost 145

9.3 Superhedging Under Illiquidity Cost

In this section, we analyze the superhedging problem under a more realistic modelaccounting for the market illiquidity. We refer to [9] for all technical details.

Following Cetin et al. [8] (CJP, hereafter), we account for the liquidity cost bymodeling the price process of this asset as a function of the exchanged volume. Wethus introduce a supply curve

S .St ; �/ ;

where � 2 R indicates the volume of the transaction, the process St D S .St ; 0/is the marginal price process defined by some given initial condition S.0/ togetherwith the Black–Scholes dynamics:

dStSt

D �dWt ; (9.35)

where as usual the prices are discounted, i.e. expressed in the numeraire defined bythe nonrisky asset, and the drift is omitted by a change of measure.

The function S : RC � R �! R is assumed to be smooth and increasing in �.S.s; �/ represents the price per share for trading of size � and marginal price s.

A trading strategy is defined by a pair .Z0;Z/ where Z0t is the position in cash

andZt is the number of shares held at each time t in the portfolio. As in the previousparagraph, we will take the processZ in the set of admissible strategies At;s definedin (9.16), whenever the problem is started at the time origin t with the initial spotprice s for the underlying asset.

To motivate the continuous-time model, we start from discrete-time tradingstrategies. Let 0 D t0 < � � � < tn D T be a partition of the time interval Œ0; T �,and denote ı .ti / WD .ti / � .ti�1/ for any function . By the self-financingcondition, it follows that

ıZ0ti

C ıZtiS .Sti ; ıZti / D 0; 1 � i � n:

Summing up these equalities, it follows from direct manipulations that

Z0T CZT ST D Z0

0 CZ0S0 �nXiD1

ŒıZtiS .Sti ; ıZti /C .Z0S0 �ZT ST /�

D Z00 CZ0S0 �

nXiD1

ŒıZti Sti C .Z0S0 �ZT ST /�

�nXiD1

ıZti ŒS .Sti ; ıZti /� Sti �

D Z00 CZ0S0 C

nXiD1

Zti�1ıSti �nXiD1

ıZti ŒS .Sti ; ıZti / � Sti � :(9.36)


Then, the continuous-time dynamics of the process

Y WD Z0 CZS

are obtained by taking limits in (9.36) as the time step of the partition shrinks tozero. The last sum term in (9.36) is the term due to the liquidity cost.

Since the function � 7�! S.s; �/ is assumed to be smooth, it follows from theform of the continuous-time process Z in (9.11) that:

Yt D Y0 CZ t

0

ZudSu �Z t

0

4

Su�.Su/dhZiu (9.37)

D Y0 CZ t

0

ZudSu �Z t

0

4

`.Su/� 2

u �2.u; Su/Sudu; (9.38)

where ` is the liquidity function defined by

`.s/ WD s

�[email protected]; 0/

��1: (9.39)

The above liquidation value of the portfolio exhibits a penalization by a linear termin � 2, with coefficient determined by the slope of the order book at the origin. Thistype of dynamics falls into the general problems analyzed in the previous section.

Remark 9.14. The supply function S.s; �/ can be inferred from the data on orderbook prices. We refer to [10] for a parametric estimation of this model on realfinancial data.

In the context of the CJP model, we ignore the illiquidity cost at the maturitydate T , and we formulate the super-hedging problem by:

V.t; s/ WD infny W Y y;ZT � g.St;sT /; P � a.s. for some Z 2 At;s

o: (9.40)

Then, the viscosity property for the value function V follows from the results of theprevious section. The next result says more as it provides uniqueness.

Theorem 9.15. Assume that V is locally bounded. Then, the super-hedging cost Vis the unique viscosity solution of the PDE problem

�@tV � 1

2�2sH

.�`/ _ .sVss/

� D 0; V .T; :/ D g (9.41)

�C � V.t; s/ � C.1C s/; .t; s/ 2 Œ0; T � � RC; for some C > 0; (9.42)

where H. / WD C 12` 2.

We refer to [9] for the proof of uniqueness. We conclude this section by somecomments.

9.3 Superhedging Under Illiquidity Cost 147

Remark 9.16. 1. The PDE (9.41) is very similar to the PDE obtained in the problemof hedging under Gamma constraints. We observe here that �` plays the samerole as the lower bound � on the Gamma of the portfolio. Therefore, the CJPmodel induces an endogenous (state-dependent) lower bound on the Gamma ofthe portfolio defined by `.

2. However, there is no counterpart in (9.41) to the upper bound � which inducedthe face-lifting of the payoff in the problem of hedging under Gamma constraints.

Chapter 10Backward SDEs and Stochastic Control

In this chapter, we introduce the notion of backward stochastic differential equation(BSDE hereafter) which allows to relate standard stochastic control to stochastictarget problems. More importantly, the general theory in this chapter will bedeveloped in the non-Markovian framework. The Markovian framework of theprevious chapters and the corresponding PDEs will be obtained under a specificconstruction. From this viewpoint, BSDEs can be viewed as the counterpart of PDEsin the non-Markovian framework.

However, by their very nature, BSDEs can only cover the subclass of standardstochastic control problems with uncontrolled diffusion, with corresponding semi-linear DPE. Therefore, a further extension is needed in order to cover the moregeneral class of fully nonlinear PDEs, as those obtained as the DPE of standardstochastic control problems. This can be achieved by means of the notion of second-order BSDEs which are very connected to second-order target problems. We referto Cheridito et al. [13] and Soner et al. [38] for this extension.

10.1 Motivation and Examples

The first appearance of BSDEs was in the early work of Bismut [6] who was extend-ing the Pontryagin maximum principle of optimality to the stochastic framework.Similar to the deterministic context, this approach introduces the so-called adjointprocess defined by a stochastic differential equation combined with a final condition.In the deterministic framework, the existence of a solution to the adjoint equationfollows from the usual theory by obvious time inversion. The main difficulty in thestochastic framework is that the adjoint process is required to be adapted to the givenfiltration, so that one cannot simply solve the existence problem by running the timeclock backward.

A systematic study of BSDEs was started by Pardoux and Peng [32]. Themotivation was also from optimal control which was an important field of interest


149

150 10 Backward SDEs and Stochastic Control

for Shige Peng. However, the natural connections with problems in financialmathematics were very quickly realized, see Elkaroui et al. [18]. Therefore, a largedevelopment of the theory was achieved in connection with financial applicationsand crucially driven by the intuition from finance.

10.1.1 The Stochastic Pontryagin Maximum Principle

Our objective in this section is to see how the notion of BSDE appears naturally inthe context of the Pontryagin maximum principle. Therefore, we are not intendingto develop any general theory about this important question, and we will not makeany effort in weakening the conditions for the main statement. We will insteadconsiderably simplify the mathematical framework in order for the main ideas tobe as transparent as possible.

Consider the stochastic control problem

V0 WD sup�2U0

J0.�/ where J0.�/ WD E�g.X�

T /�;

the set of control processes U0 is defined as in Sect. 3.1, and the controlled stateprocess is defined by some initial date X0 and the SDE with random coefficients:

dX�t D b.t; X�

t ; �t /dt C �.t; X�t ; �t /dWt:

Observe that we are not emphasizing the time origin and the position of the statevariable X at the time origin. This is a major difference between the dynamicprogramming approach, developed by the American school, and the Pontryaginmaximum principle approach of the Russian school.

For every u 2 U , we define:

Lu.t; x; y; z/ WD b.t; x; u/ � y C Tr��.t; x; u/Tz

�;

so that

b.t; x; u/ D @Lu.t; x; y; z/

@yand �.t; x; u/ D @Lu.t; x; y; z/

@z:

We also introduce the function

`.t; x; y; z/ WD supu2U

Lu.t; x; y; z/;

and we will denote by H2 the space of all F�progressively measurable processes

with finite L2 .Œ0; T � � �; dt ˝ dP/ �norm.

10.1 Motivation and Examples 151

Theorem 10.1. Let O� 2 U0 be such that

(i) There is a solution . OY ; OZ/ in H2 of the backward stochastic differential

equation:

d OYt D �rxLO�t .t; OXt ; OYt ; OZt /dt C Zt dWt ; and OYT D rg. OXT / (10.1)

where OX WD X O�(ii) O� satisfies the maximum principle:

LO�t .t; OXt; OYt ; OZt / D `.t; OXt ; OYt ; OZt /: (10.2)

(iii) The functions g and `.t; :; y; z/ are concave, for fixed t; y; z, and

rxLO�t .t; OXt; OYt ; OZt / D rx`.t; OXt ; OYt ; OZt /: (10.3)

Then V0 D J0. O�/, i.e., O� is an optimal control for the problem V0.

Proof. For an arbitrary � 2 U0, we compute that

J0. O�/ � J0.�/ D E

hg. OXT / � g.X�

T /i

� E

h. OXT � X�

T / � rg. OXT /i

D E

h. OXT � X�

T / � OYT

i

by the concavity assumption on g. Using the dynamics of OX and OY, this provides

J0. O�/ � J0.�/ � E

�Z T

0

d˚. OXT � X�

T / � OYT

��

D E

�Z T

0

�b.t; OXt; O�t / � b.t; X�

t ; �t /� � OYt dt

�. OXt � X�t / � rxLO�t .t; OXt; OYt ; OZt /dt

CTr

��.t; OXt ; O�t / � �.t; X�

t ; �t /�T OZt

�dt

�

D E

h Z T

0

�LO�t .t; OXt ; OYt ; OZt / � L�t .t; Xt ; OYt ; OZt /

�. OXt � X�t / � rxLO�t .t; OXt; OYt ; OZt /

�dti;

where the diffusion terms have zero expectations because the processes OY and OZare in H

2. By Conditions (ii) and (iii), this implies that


J0. O�/ � J0.�/ � E

�Z T

0

�`.t; OXt; OYt ; OZt / � `.t; Xt ; OYt ; OZt /

�. OXt � X�t / � rx`.t; OXt ; OYt ; OZt /

�dt

�

� 0

by the concavity assumption on `. utLet us comment on the conditions of the previous theorem.

– Condition (ii) provides a feedback definition to O�. In particular, O�t is a functionof .t; OXt ; OYt ; OZt /. As a consequence, the forward SDE defining OX depends on thebackward component . OY ; OZ/. This is a situation of forward-backward stochasticdifferential equation which will not be discussed in these notes.

– Condition (10.3) in (iii) is satisfied under natural smoothness conditions. In theeconomic literature, this is known as the envelope theorem.

– Condition (i) states the existence of a solution to the BSDE (10.1), which will bethe main focus of the subsequent section.

10.1.2 BSDEs and Stochastic Target Problems

Let us go back to a subclass of the stochastic target problems studied in Chap. 8defined by taking the state process X independent of the control Z which is assumedto take values in R

d . For simplicity, let X D W . Then the stochastic target problemis defined by

V0 WD inf˚Y0 W Y Z

T � g.WT /; P � a.s. for some Z 2 H2�

;

where the controlled process Y satisfies the dynamics:

dY Zt D b.t; Wt ; Yt ; Zt /dt C Zt � dWt : (10.4)

If existence holds for the latter problem, then there would exist a pair .Y; Z/ in H2

such that

Y0 CZ T

0

�b.t; Wt ; Yt ; Zt /dt C Zt � dWt

� � g.WT /; P � a.s.

If in addition equality holds in the latter inequality, then .Y; Z/ is a solution of theBSDE defined by (10.4) and the terminal condition YT D g.WT /, P�a.s.

10.1 Motivation and Examples 153

10.1.3 BSDEs and Finance

In the Black–Scholes model, we know that any derivative security can be perfectlyhedged. The corresponding superhedging problem reduces to a hedging problem,and an optimal hedging portfolio exists and is determined by the martingalerepresentation theorem.

In fact, this goes beyond the Markovian framework to which the stochastic targetproblems are restricted. To see this, consider a financial market with interest rateprocess frt ; t � 0g and d risky assets with price process defined by

dSt D St ? .�t dt C �t dWt/:

Then, under the self-financing condition, the liquidation value of the portfolio isdefined by

dY �t D rtY

�t dt C �t �t .dWt C �t dt/; (10.5)

where the risk premium process �t WD ��1t .�t � rt 1/ is assumed to be well defined,

and the control process �t denotes the vector of holdings amounts in the d riskyassets at each point in time.

Now let G be a random variable indicating the random payoff of a contract. G

is called a contingent claim. The hedging problem of G consists in searching for aportfolio strategy O� such that

Y O�T D G; P � a.s. (10.6)

We are then reduced to a problem of solving the BSDE (10.5)–(10.6). This problemcan be solved very easily if the process � is so that the process fWt C

R t

0�sds; t � 0g

is a Brownian motion under the so-called equivalent probability measure Q. Underthis condition, it suffices to get rid of the linear term in (10.5) by discounting, thenO� is obtained by the martingale representation theorem in the present Brownianfiltration under the equivalent measure Q.

We finally provide an example where the dependence of Y in the control variableZ is nonlinear. The easiest example is to consider a financial market with differentlending and borrowing rates r t � rt . Then the dynamics of liquidation value of theportfolio (10.5) is replaced by the following SDE:

dYt D �t � �t .dWt C �t dt/.Yt � �t � 1/Cr t � .Yt � �t � 1/�rt : (10.7)

As a consequence of the general result of the subsequent section, we will obtainthe existence of a hedging process O� such that the corresponding liquidation valuesatisfies (10.7) together with the hedging requirement (10.6).


10.2 Wellposedness of BSDEs

Throughout this section, we consider a d�dimensional Brownian motion W on acomplete probability space .�;F ;P/, and we denote by F D F

W the correspondingaugmented filtration.

Given two integers n; d 2 N, we consider the mapping

f W Œ0; T � � � � Rn � R

n�d �! R

that we assume to be P ˝ B.RnCnd /�measurable, where P denotes the ��algebragenerated by predictable processes. In other words, for every fixed .y; z/ 2 R

n �R

n�d , the process fft .y; z/; t 2 Œ0; T �g is F�predictable.Our interest is on the BSDE:

dYt D �ft .Yt ; Zt /dt C Zt dWt and YT D ; P � a.s. (10.8)

where is some given FT �measurable r.v. with values in Rn.

We will refer to (10.8) as BSDE.f; /. The map f is called the generator. Wemay also re-write the BSDE (10.8) in the integrated form:

Yt D CZ T

t

fs.Ys; Zs/ds �Z T

t

ZsdWs; t � T ; P � a.s. (10.9)

10.2.1 Martingale Representation for Zero Generator

When the generator f � 0, the BSDE problem reduces to the martingalerepresentation theorem in the present Brownian filtration. More precisely, forevery 2 L

2.Rn;FT /, there is a unique pair process .Y; Z/ in H2.Rn � R

n�d /

satisfying (10.8):

Yt WD EŒjFt � D EŒ� CZ t

0

ZsdWs

D �Z T

t

ZsdWs:

Here, for a subset E of Rk , k 2 N, we denoted by H

2.E/ the collection of allF�progressively measurable L2.Œ0; T � � �; Leb ˝ P/�processes with values in E .We shall frequently simply write H2 keeping the reference to E implicit.

Let us notice that Y is a uniformly integrable martingale. Moreover, by theDoob’s maximal inequality, we have

10.2 Wellposedness of BSDEs 155

kY k2S2 WD E

"

supt�T

jYt j2#

� 4E�jYT j2� D 4kZk2

H2 : (10.10)

Hence, the process Y is in the space of continuous processes with finite S2�norm.

10.2.2 BSDEs with Affine Generator

We next consider a scalar BSDE (n D 1) with generator

ft .y; z/ WD at C bt y C ct � z; (10.11)

where a; b; andc are F�progressively measurable processes with values in R, R,

and Rd , respectively. We also assume that b; c are bounded and E

hR T

0jat j2dt

i

< 1.This case is easily handled by reducing to the zero generator case. However, it

will play a crucial role for the understanding of BSDEs with generator quadratic inz, which will be the focus of the next chapter.

First, by introducing the equivalent probability Q � P defined by the density

dQ

dPD exp

Z T

0

ct � dWt � 1

2

Z T

0

jct j2dt

;

it follows from the Girsanov theorem that the process Bt WD Wt � R t

0csds defines a

Brownian motion under Q. By formulating the BSDE under Q,

dYt D �.at C btYt /dt C Zt � dBt;

we have reduced to the case where the generator does not depend on z. We next getrid of the linear term in y by introducing

Y t WD Yt eR t

0 bsds so that dY t D �at eR t

0 bsdsdt C Zt eR t

0 bsdsdBt :

Finally, defining

Y t WD Y t CZ t

0

aueR u

0 bsdsdu;

we arrive at a BSDE with zero generator for Y t which can be solved by themartingale representation theorem under the equivalent probability measure Q.


Of course, one can also express the solution under P:

Yt D E

� t

T CZ T

t

ts asds

ˇˇˇFt

�; t � T;

where

ts WD exp

Z s

t

budu � 1

2

Z s

t

jcuj2du CZ s

t

cu � dWu

; 0 � t � s � T: (10.12)

10.2.3 The Main Existence and Uniqueness Result

The following result was proved by Pardoux and Peng [32].

Theorem 10.2. Assume that fft .0; 0/; t 2 Œ0; T �g 2 H2 and, for some constant

C > 0,

ˇˇft .y; z/ � ft .y

0; z0/ˇˇ � C.jy � y0j C jz � z0j/; dt ˝ dP � a.s.

for all t 2 Œ0; T � and .y; z/; .y0; z0/ 2 Rn � R

n�d . Then, for every 2 L2, there is a

unique solution .Y; Z/ 2 S2 � H2 to the BSDE.f; /.

Proof. Denote S D .Y; Z/, and introduce the equivalent norm in the correspondingH

2 space:

kSk˛ WD E

�Z T

0

e˛t .jYt j2 C jZt j2/dt

�;

where ˛ will be fixed later. We consider the operator

� W s D .y; z/ 2 H2 7�! Ss D .Y s; Zs/

defined by:

Y st D C

Z T

t

fu.yu; zu/du �Z T

t

Zsu � dWu; t � T:

1. First, since jfu.yu; zu/j � jfu.0; 0/j C C.jyuj C jzuj/, we see that the processffu.yu; zu/; u � T g is in H

2. Then Ss is well defined by the martingalerepresentation theorem and Ss D �.s/ 2 H

2.2. For s; s0 2 H

2, denote ıs WD s � s0, ıS WD Ss � Ss0

and ıf WD ft .Ss/ � ft .S

s0

/.Since ıYT D 0, it follows from Ito’s formula that

10.2 Wellposedness of BSDEs 157

e˛t jıYt j2 CZ T

t

e˛ujıZuj2du DZ T

t

e˛u�2ıYu � ıfu � ˛jıYuj2� du

�2

Z T

t

e˛u.ıZu/TıYu � dWu:

In the remaining part of this step, we prove that

M: WDZ :

0

e˛u.ıZu/TıYu � dWu is a uniformly integrable martingale, (10.13)

so that we deduce from the previous equality that

E

�e˛t jıYt j2 C

Z T

t

e˛ujıZuj2du

�D E

�Z T

t

e˛u�2ıYu � ıfu � ˛jıYuj2� du

�:

(10.14)

To prove (10.13), we verify that supt�T jMt j 2 L1. Indeed, by the Burkholder-

Davis-Gundy inequality, we have

E

hsupt�T

jMt ji

� CE

"Z T

0

e2˛ujıYuj2jıZuj2du

1=2#

� C 0E

"

supu�T

jıYujZ T

0

jıZuj2du

1=2#

� C 0

2

E

"

supu�T

jıYuj2#

C E

�Z T

0

jıZuj2du

�!

< 1:

3. We now continue estimating (10.14) by using the Lipschitz property of thegenerator:

E

he˛t jıYt j2 C

Z T

t

e˛ujıZuj2dui

� E

h Z T

t

e˛u��˛jıYuj2 C C 2jıYuj.jıyuj C jızuj/� du

i

� E

�Z T

t

e˛u��˛jıYuj2 C C

�"2jıYuj2C"�2.jıyuj C jızuj/2

��du

�

for any " > 0. Choosing C "2 D ˛, we obtain:

E

�e˛t jıYt j2 C

Z T

t

e˛ujıZuj2du

�� E

�Z T

t

e˛u C 2

˛.jıyuj C jızuj/2du

�

� 2C 2

˛kısk2

˛:


This provides

kıZk2˛ � 2

C 2

˛kısk2

˛ and kıY k2˛dt � 2

C 2T

˛kısk2

˛

where we abused notation by writing kıY k˛ and kıZk˛ although these processesdo not have the dimension required by the definition. Finally, these two estimatesimply that

kıSk˛ �r

2C 2

˛.1 C T /kısk˛:

By choosing ˛ > 2.1 C T /C 2, it follows that the map � is a contraction on H2,

and that there is a unique fixed point.4. It remain, to prove that Y 2 S2. This is easily obtained by first estimating:

E

"

supt�T

jYt j2#

� C

jY0j2 C E

�Z T

0

jft .Yt ; Zt /j2dt

�

C E

"

supt�T

ˇˇˇZ t

0

Zs � dWs

ˇˇˇ2

#!

;

and then using the Lipschitz property of the generator and the Burkholder-Davis-Gundy inequality. ut

Remark 10.3. Consider the Picard iterations:

.Y 0; Z0/ D .0; 0/; and

Y kC1t D C

Z T

t

fs.Yks ; Zk

s /ds CZ T

t

ZkC1s � dWs:

Given .Y k; Zk/, the next step .Y kC1; ZkC1/ is defined by means of the martingalerepresentation theorem. Then, Sk D .Y k; Zk/ �! .Y; Z/ in H

2 as k ! 1.Moreover, since

kSkk˛ �

2C 2

˛.1 C T /

k

;

it follows thatP

k kSkk˛ < 1, and we conclude by the Borel–Cantelli lemma thatthe convergence .Y k; Zk/ �! .Y; Z/ also holds dt ˝ dP�a.s.

10.3 Comparison and Stability 159

10.3 Comparison and Stability

Theorem 10.4. Let n D 1, and let .Y i ; Zi/ be the solution of BSDE.f i ; i / forsome pair .i ; f i / satisfying the conditions of Theorem 10.2, i D 0; 1. Assume that

1 � 0 and f 1t .Y 0

t ; Z0t / � f 0

t .Y 0t ; Z0

t /; dt ˝ dP � a.s. (10.15)

Then Y 1t � Y 0

t , t 2 Œ0; T �, P�a.s.

Proof. We denote

ıY WD Y 1 � Y 0; ıZ WD Z1 � Z0; ı0f WD f 1.Y 0; Z0/ � f 0.Y 0; Z0/;

and we compute that

d.ıYt / D � .˛t ıYt C ˇt � ıZt C ı0ft / dt C ıZt � dWt ; (10.16)

where

˛t WD f 1t .Y 1

t ; Z1t / � f 1

t .Y 0t ; Z1

t /

ıYt

1fıYt ¤0g;

and, for j D 1; : : : ; d;

ˇjt WD f 1

t

�Y 0

t ; Z1t ˚j �1 Z0

t

�� f 1t

�Y 0

t ; Z1t ˚j Z0

t

�

ıZ0;jt

1fıZ0;jt ¤0g;

where ıZ0;j denotes the j -th component of ıZ0, and for every z0; z1 2 Rd , z1 ˚j

z0 WD �z1;1; : : : ; z1;j ; z0;j C1; : : : ; z0;d

�for 0 < j < d , z1 ˚0 z0 WD z0, z1 ˚d z0 WD z1.

Since f 1 is Lipschitz-continuous, the processes ˛ and ˇ are bounded. Solvingthe linear BSDE (10.16) as in Sect. 10.2.2, we get

ıYt D E

� t

T ıYT CZ T

t

tu ı0fudu

ˇˇˇFt

�; t � T;

where the process t is defined as in (10.12) with .ı0f; ˛; ˇ/ substituted to .a; b; c/.Then Condition (10.15) implies that ıY � 0, P�a.s. ut

Our next result compares the difference in absolute value between the solutionsof the two BSDEs and provides a bound which depends on the difference betweenthe corresponding final datum and the generators. In particular, this bound providesa transparent information about the nature of conditions needed to pass to limitswith BSDEs.


Theorem 10.5. Let .Y i ; Zi / be the solution of BSDE.f i ; i / for some pair .f i ; i /

satisfying the conditions of Theorem 10.2, i D 0; 1. Then

kY 1 � Y 0k2S2 C kZ1 � Z0k2

H2 � C�k1 � 0k2

L2 C k.f 1 � f 0/.Y 0; Z0/k2H2

�;

where C is a constant depending only on T and the Lipschitz constant of f 1.

Proof. We denote ı WD 1 �0, ıY WD Y 1 �Y 0, ıf WD f 1.Y 1; Z1/�f 0.Y 0; Z0/,and �f WD f 1 � f 0. Given a constant ˇ to be fixed later, we compute by Ito’sformula that

eˇt jıYt j2 D eˇT jıj2 CZ T

t

eˇu�2ıYu � ıfu � jıZuj2 � ˇjıYuj2� du

C2

Z T

t

eˇuıZTu ıYu � dWu:

By the same argument as in the proof of Theorem 10.2, we see that the stochasticintegral term has zero expectation. Then

eˇt jıYt j2 D Et

�eˇT jıj2 C

Z T

t

eˇu�2ıYu � ıfu � jıZuj2 � ˇjıYuj2� du

�;

(10.17)

where Et WD EŒ:jFt �. We now estimate that, for any " > 0,

2ıYu � ıfu � "�1jıYuj2 C "jıfuj2

� "�1jıYuj2 C "�C.jıYuj C jıZuj/ C j�fu.Y 0

u ; Z0u/j�2

� "�1jıYuj2 C 3"�C 2.jıYuj2 C jıZuj2/ C j�fu.Y 0

n ; Z0u/j2�:

We then choose " WD 1=.6C 2/ and ˇ WD 3"C 2 C "�1, and plug the latter estimatein (10.17). This provides:

eˇt jıYt j2C1

2Et

�Z T

t

jıZuj2du

�� Et

�eˇT jıj2C 1

2C 2

Z T

0

eˇuj�fu.Y 0u ; Z0

u /j2du

�;

which implies the required inequality by taking the supremum over t 2 Œ0; T � andusing the Doob’s maximal inequality for the martingale fEt ŒeˇT jıj2�; t � T g. ut

10.4 BSDEs and Stochastic Control

We now turn to the question of controlling the solution of a family of BSDEs in thescalar case n D 1. Let .f�; �/�2U be a family of coefficients, where U is some givenset of controls. We assume that the coefficients .f�; �/�2U satisfy the conditions

10.4 BSDEs and Stochastic Control 161

of the existence and uniqueness of Theorem 10.2, and we consider the followingstochastic control problem:

V0 WD sup�2U

Y �0 ; (10.18)

where .Y �; Z�/ is the solution of BSDE.f �; �/.The above stochastic control problem boils down to the standard control

problems of Sect. 3.1 when the generators f ˛ are all zero. When the generatorsf � are affine in .y; z/, the problem (10.18) can also be recasted in the standardframework, by discounting and change of measure.

The following easy result shows that the above maximization problem can besolved by maximizing the coefficients .˛; f ˛/:

ft .y; z/ WD ess sup�2U

f �t .y; z/; WD ess sup

�2U�: (10.19)

The notion of essential supremum is recalled in the Appendix of this chapter.We will assume that the coefficients .f; / satisfy the conditions of the existenceresult of Theorem 10.2, and we will denote by .Y; Z/ the corresponding solution.

A careful examination of the statement below shows a great similarity with theverification result in stochastic control. In the present non-Markovian framework,this remarkable observation shows that the notion of BSDEs allows to mimic thestochastic control methods developed in the Markovian framework of the previouschapters.

Proposition 10.6. Assume that the coefficients .f; / and .f�; �/ satisfy the condi-tions of Theorem 10.2, for all � 2 U . Assume further that there exists some O� 2 Usuch that

ft .y; z/ D f O�.y; z/ and D O�:

Then V0 D Y O�0 and Yt D ess sup�2U Y �

t , t 2 Œ0; T �, P�a.s.

Proof. The P�a.s. inequality Y � Y � , for all � 2 U , is a direct consequence of thecomparison result of Theorem 10.4. Hence, Yt � sup�2U Y �

t , P�a.s. To conclude,we notice that Y and Y O� are two solutions of the same BSDE, and therefore mustcoincide, by uniqueness. ut

The next result characterizes the solution of a standard stochastic control problemin terms of a BSDE. Here, again, we emphasize that, in the present non-Markovianframework, the BSDE is playing the role of the dynamic programming equationwhose scope is restricted to the Markovian case.

Let

U0 WD inf�2U E

P�

�ˇ�

0;T � CZ T

0

ˇ�u;T `u.�u/du

�;


where

dP�

dP

ˇˇˇˇFT

WD eR T

0 �t .�t /�dWt � 12

R T0 j�t .�t /j2dt and ˇ�

t;T WD e� R Tt ku.�u/du:

We assume that all coefficients involved in the above expression satisfy the requiredconditions for the problem to be well defined.

We first notice that for every � 2 U , the process

Y �t WD E

P�

�ˇ�

t;T � CZ T

t

ˇ�u;T `u.�u/du

ˇˇFt

�; t 2 Œ0; T �;

is the first component of the solution .Y �; Z�/ of the affine BSDE:

dY �t D �f �

t .Y �t ; Z�

t /dt C Z�t dWt; Y �

T D �;

with f �t .y; z/ WD `t .�t / � kt .�t /y C �t .�t /z. In view of this observation, the

following result is a direct application of Proposition 10.6.

Proposition 10.7. Assume that the coefficients

WD ess sup�2U

� and ft .y; z/ WD ess sup�2U

f �t .y; z/

satisfy the conditions of Theorem 10.2, and let .Y; Z/ be the corresponding solution.Then U0 D Y0.

10.5 BSDEs and Semilinear PDEs

In this section, we specialize the discussion to the so-called Markovian BSDEs inthe one-dimensional case n D 1. This class of BSDEs corresponds to the case where

ft .y; z/ D F.t; Xt ; y; z/ and D g.XT /;

where F W Œ0; T � � Rd � R � R

d �! R and g W Rd �! R are measurable, and X

is a Markovian diffusion process defined by some initial data X0 and the SDE:

dXt D �.t; Xt /dt C �.t; Xt /dWt: (10.20)

Here � and � are continuous and satisfy the usual Lipschitz and linear growthconditions in order to ensure existence and uniqueness of a strong solution to theSDE (10.20), and

F; g have polynomial growth in x;

and F is uniformly Lipschitz in .y; z/:

10.5 BSDEs and Semilinear PDEs 163

Then, it follows from Theorem 10.2 that the above Markovian BSDE has a uniquesolution.

We next move the time origin by considering the solution fXt;xs ; s � tg of (10.20)

with initial data Xt;xt D x. The corresponding solution of the BSDE

dYs D �F.s; Xt;xs ; Ys; Zs/ds C ZsdWs; YT D g

�X

t;xT

�(10.21)

will be denoted by .Y t;x ; Zt;x/.

Proposition 10.8. The process f�Y t;xs ; Zt;x

s

�; s 2 Œt; T �g is adapted to the filtration

F ts WD � .Wu � Wt ; u 2 Œt; s�/ ; s 2 Œt; T �:

In particular, u.t; x/ WD Yt;xt is a deterministic function and

Y t;xs D Y s;X

t;xs

s D u�s; Xt;x

s

�; for all s 2 Œt; T �; P � a.s.

Proof. The first claim is obvious, and the second one follows from the fact thatXt;x

r D Xs;Xt;xs

r . utProposition 10.9. Let u be the function defined in Proposition 10.8, and assumethat u 2 C 1;2.Œ0; T /;Rd /. Then

� @t u � � � Du � 1

2TrŒ��TD2u� � f .:; u; �TDu/ D 0 on Œ0; T / � R

d : (10.22)

Proof. This an easy application of Ito’s formula together with the usual localizationtechnique. ut

By weakening the interpretation of the PDE (10.22) to the sense of viscositysolutions, we may drop the regularity condition on the function u in the latterstatement. We formulate this result in the following exercise.

Exercise 10.10. Show that the function u of Proposition 10.8 is a viscosity solutionof the semilinear PDE (10.22), i.e., u� and u� are viscosity supersolution andsubsolutions of (10.22), respectively.

We conclude this chapter by nonlinear version of the Feynman–Kac formula.

Theorem 10.11. Let v 2 C 1;2.Œ0; T /;Rd / be a solution of the semilinearPDE (10.22) with polynomially growing v and �TDv. Then

v.t; x/ D Y t;xt for all .t; x/ 2 Œ0; T � � R

d ;

where .Y t;x; Zt;x/ is the solution of the BSDE (10.21).

Proof. For fixed .t; x/, denote Ys WD v.s; Xt;xs / and Zs WD �T.s; Xt;x

s /. Then, itfollows from Ito’s formula that .Y; Z/ solves (10.21). From the polynomial growthon v and Dv, we see that the processes Y and Z are both in H

2. Then they coincidewith the unique solution of (10.21). ut


10.6 Appendix: Essential Supremum

The notion of essential supremum has been introduced in probability in order to facethe problem of maximizing random variables over an infinite family Z . The problemarises when Z is not countable because then the supremum is not measurable, ingeneral.

Theorem 10.12. Let Z be a family of r.v. Z W � �! R [ f1g on a probabilityspace .�;F ;P/. Then there exists a unique (a.s.) r.v. NZ W � �! R [ f1g suchthat

(a) NZ � Z, a.s. for all Z 2 Z .(b) For all r.v. Z0 satisfying (a), we have NZ � Z0, a.s.

Moreover, there exists a sequence .Zn/n2N Z such that NZ D supn2N Zn.The r.v. NZ is called the essential supremum of the family Z and denoted by ess supZ .

Proof. The uniqueness of NZ is an immediate consequence of (b). To prove exis-tence, we consider the set D of all countable subsets of Z . For all D 2 D, we defineZD WD supfZ W Z 2 Dg, and we introduce the r.v. WD supfEŒZD� W D 2 Dg.

1. We first prove that there exists D� 2 D such that D EŒZD� �. To see this, let.Dn/n D be a maximizing sequence, i.e., EŒZDn � �! , then D� WD [nDn 2D satisfies EŒZD� � D . We denote NZ WD ZD� .

2. It is clear that the r.v. NZ satisfies (b). To prove that property (a) holds true, weconsider an arbitrary Z 2 Z together with the countable family D WD D� [fZg D. Then ZD D Z _ NZ, and D EŒ NZ� � EŒZ _ NZ� � . Consequently,Z _ NZ D NZ, and Z � NZ, a.s. ut

Chapter 11Quadratic Backward SDEs

In this chapter, we consider an extension of the notion of BSDEs to the case wherethe dependence of the generator in the variable z has quadratic growth. In theMarkovian case, this corresponds to a problem of second-order semilinear PDEwith quadratic growth in the gradient term. The first existence and uniquenessresult in this context was established by M. Kobylanski in her Ph.D. thesis byadapting some previously established PDE techniques to the non-Markovian BSDEframework. In this chapter, we present an alternative argument introduced recentlyby Tevzadze [39].

Quadratic BSDEs turn out to play an important role in the applications, and theextension of this section is needed in order to analyze the problem of portfoliooptimization under portfolio constraints.

We shall consider throughout this chapter the BSDE

Yt D � CZ T

t

fs.Zs/ds �Z T

t

Zs � dWs (11.1)

where � is a bounded FT �measurable r.v. and f W Œ0; T � � � � Rd �! R is

P ˝ B.Rd /�measurable and satisfies a quadratic growth condition:

k�k1 < 1 and jft .z/j � C.1 C jzj2/ for some constant C > 0: (11.2)

We could have included a Lipschitz dependence of the generator on the variabley without altering the results of this chapter. However, for exposition clarity andtransparency, we drop this dependence in order to concentrate on the main difficulty,namely, the quadratic growth in z.


165

166 11 Quadratic Backward SDEs

11.1 A Priori Estimates and Uniqueness

In this section, we prove two easy results. First, we show the connection between theboundedness of the component Y of the solution and the bounded mean oscillation(BMO) property for the martingale part

R :

0Zt � dWt . Then, we prove uniqueness in

this class.

11.1.1 A Priori Estimates for Bounded Y

We denote by M2 the collection of all P�square integrable martingales on the timeinterval Œ0; T �. We first introduce the so-called class of martingales with BMO:

BMO WD ˚M 2 M2 W kM kBMO < 1�

;

where

kM kBMO WD sup�2T T

0

kEŒhM iT � hM i� jF� �k1 :

Here, T T0 is the collection of all stopping times, and hM i denotes the quadratic

variation process of M . We will be essentially working with square integrablemartingales of the form M D R :

0�sdWs. The following definition introduces an

abuse of notation which will be convenient for our presentation.

Definition 11.1. A process � 2 H2 is said to be a BMO martingale generator if

k�kH

2BMO

WD �� R :

0 �s � dWs

��BMO < 1:

We denote by H2BMO WD ˚

� 2 H2 W k�k

H2BMO

< 1�.

For this class of martingales, we can rewrite the BMO norm by the Ito isometryinto

k�k2

H2BMO

WD sup�2T T

0

��E�Z T

�

j�sj2dsˇF�

��1:

The following result shows why this notion is important in the context ofquadratic BSDEs.

Lemma 11.2. Let .Y; Z/ be a solution of the quadratic BSDE (11.1) (in particular,Z 2 H

2loc) with generator f satisfying (11.2). Assume that the process Y is

bounded. Then Z 2 H2BMO.

Proof. Let .�n/n�1 � T T0 be a localizing sequence of the local martingale

R :

0Zs �

dWs. By Ito’s formula together with the boundedness of Y , we have for any � 2 T T0 :

11.1 A Priori Estimates and Uniqueness 167

eˇkY k1 � eˇY�n � eˇY� DZ �n

�

ˇeˇYs

��1

2ˇjZs j2 � fs.Zs/

�ds C Zs � dWs

�:

By the Doob’s optional sampling theorem, this provides

ˇ2

2E

�Z �n

�

eˇYs jZs j2dsˇˇF�

�� eˇkY k1 C ˇE

�Z �n

�

eˇYs fs.Zs/dsˇˇF�

�

� .1 C ˇC T /eˇkY k1 C ˇCE

�Z �n

�


�:

Then, setting ˇ D 4C , it follows that

e�ˇkY k1E

�Z T

�

jZsj2dsˇˇF�

�� E

�Z T

�


�

D limn!1 " E

�Z �n

�

eˇYs jZsj2dsˇˇF�

�

� 1 C 4C 2T

4C 2eˇkY k1 ;

which provides the required result by the arbitrariness of � 2 T T0 . ut

11.1.2 Some Properties of BMO Martingales

In this section, we list without proof some properties of the space BMO. We refer tothe book of Kazamaki [25] for a complete presentation on this topic.

1. The set BMO is a Banach space.2. M 2 BMO if and only if

RHdM 2 BMO for all bounded progressively

measurable process H .3. If M 2 BMO, then

(a) The process E.M / WD eM� 12 hM i is a uniformly integrable martingale.

(b) The process M � hM i is a BMO martingale under the equivalent measureE.M / � P.

(c) E.M / 2 Lr for some r > 1.

4. For � 2 H2BMO, we have

E

"�Z T

0

j�sj2ds

�p#

� 2pŠ�4k�k2

H2BMO

p

for all p � 1:

In our subsequent analysis, we shall only make use of the Properties 1 and 3.


11.1.3 Uniqueness

We now introduce the main condition for the derivation of the existence anduniqueness result.

Assumption 11.3. The quadratic generator f is C 2 in z, and there are constants�1; �2 such that

jDzft .z/j � �1.1 C jzj/; jD2zzft .z/j � �2 for all .t; !; z/ 2 Œ0; T � � � � R

d :

Lemma 11.4. Let Assumption 11.3 hold true. Then, there exists a bounded progres-sively measurable process � such that for all t 2 Œ0; T �; z; z0 2 R

d

ˇft .z/ � ft .z

0/ � �t � .z � z0/ˇ � �2jz � z0j jzj C jz0j� ; P � a.s. (11.3)

Proof. Since f is C 2 in z, we introduce the process �t WD Dzft .0/ which isbounded by �1, according to Assumption 11.3. By the mean value theorem, wecompute that, for some constant � D �.!/ 2 Œ0; 1�,

ˇft .z/ � ft .z

0/ � �t � .z � z0/ˇ D ˇ

Dzft .�z C .1 � �/z0/ � �t

ˇ jz � z0j� �2j�z C .1 � �/z0j jz � z0j;

by the bound on D2zzft .z/ in Assumption 11.3. The required result follows from the

trivial inequality j�z C .1 � �/z0j � jzj C jz0j. utWe are now ready for the proof of the uniqueness result. As in the Lipschitz case,

we have the following comparison result which implies uniqueness.

Theorem 11.5. Let f 0; f 1 be two quadratic generators satisfying (11.2). Assumefurther that f 1 satisfies Assumption 11.3. Let .Y i ; Zi /, i D 0; 1, be two boundedsolutions of (11.1) with coefficients .f i ; �i /. Assume that

�1 � �0 and f 1t .Z0

t / � f 0t .Z0

t /; t 2 Œ0; T �; P � a.s.

Then Y 1 � Y 0, P�a.s.

Proof. We denote ı� WD �1 � �0, ıY WD Y 1 � Y 0, ıZ WD Z1 � Z0, and ıf WDf 1.Z1/ � f 0.Z0/. Then, it follows from Lemma 11.4 that

ıYt D ı� �Z T

t

ıZs � dWs CZ T

t

ıfsds

� ı� �Z T

t

ıZs � dWs CZ T

t

.f 1 � f 0/.Z0s /ds

CZ T

t

f 1.Z1

s / � f 1.Z0s /

�ds

11.2 Existence 169

� ı� �Z T

t

ıZs � dWs CZ T

t

.f 1 � f 0/.Z0s /ds

CZ T

t

��s � .Z1

s � Z0s / � �2jZ1

s � Z0s j.jZ0

s j C jZ1s j/

ds

D ı� �Z T

t

ıZs � .dWs � sds/ CZ T

t

.f 1 � f 0/.Z0s /ds;

where � is the bounded process introduced in Lemma 11.4 and the process isdefined by

s WD �s � �2

jZ0s j C jZ1

s jjZ1

s � Z0s j .Z1

s � Z0s /1fZ1

s �Z0s ¤0g; s 2 Œt; T �:

Since Y 0 and Y 1 are bounded, and both generators f 0; f 1 satisfy Condition (11.2),it follows from Lemma 11.2 that Z0 and Z1 are in H

2BMO. Hence, 2 H

2BMO, and

by Property 3 of BMO martingales, we deduce that the process W: � R :

0 sds is aBrownian motion under an equivalent probability measure Q. Taking conditionalexpectations under Q then provides

ıYt � EQ

t

�ı� C

Z T

t

.f 1 � f 0/.Z0s /ds

�; a.s.

which implies the required comparison result. ut

11.2 Existence

In this section, we prove existence of a solution to the quadratic BSDE in twosteps. We first prove existence (and uniqueness) by a fixed point argument whenthe final data � is bounded by some constant depending on the generator f and thematurity T . In the second step, we decompose the final data as � D Pn

iD1 �i with�i is sufficiently small so that the existence result of the first step applies. Then,we construct a solution of the quadratic BSDE with final data � by adding thesesolutions.

11.2.1 Existence for Small Final Condition

In this subsection, we prove an existence and uniqueness result for the quadraticBSDE (11.1) under Condition (11.3) with � � 0. Recall that (11.3) was implied byAssumption 11.3.


Theorem 11.6. Assume that the generator f satisfies

ft .0/ D 0 andˇft .z/ � ft .z

0/ˇ � �2jz � z0j jzj C jz0j� ; P � a.s. (11.4)

Then, for every FT �measurable r.v. � with k�kL1 � .64�2/�1, there exists a unique

solution .Y; Z/ to the quadratic BSDE (11.1) with

kY k2S1 C kZk2

H2BMO

� .16�2/�2 :

Proof. Consider the map ˚ W .y; z/ 2 S1 � H2BMO 7�! S D .Y; Z/ defined by:

Yt D � CZ T

t

fs.zs/ds �Z T

t

Zs � dWs; t 2 Œ0; T �; P � a.s.

The existence of the pair .Y; Z/ D ˚.y; z/ 2 H2 is justified by the martingale

representation theorem together with Property 4 of BMO martingales which ensuresthat the process f .Z/ is in H

2.To obtain the required result, we will prove that ˚ is a contracting mapping on

S1 � H2BMO when � has a small L1�norm as in the statement of the theorem.

1. In this step, we prove that

.Y; Z/ D ˚.y; z/ 2 S1 � H2:

First, we estimate that

jYt j DˇˇEt

�� C

Z T

t

fs.zs/ds

�ˇˇ

� k�k1 C C�T C kzk

H2BMO

;

proving that the process Y is bounded. We next calculate by Ito’s formula that,for every stopping time � 2 T T

0 ,

jY� j2 C E�

�Z T

�

jZs j2ds

�D E�

hj�j2 C

Z T

�

2Ysfs.zs/dsi

� k�k2L1 C 2kY kS1E�

�Z T

�

jfs.zs/jds

�;

where E� Œ:� D EŒ:jF� � and, similar to the proof of Theorem 10.2, the expectationof the stochastic integral vanishes by Property 4 of BMO martingales.

11.2 Existence 171

By the trivial inequality 2ab � 14a2 C 4b2, it follows from the last inequality

that:

jY� j2 C E�

�Z T

�

jZsj2ds

�� k�k2

L1 C 1

4kY k2

S1 C 4

�E�

�Z T

�

jfs.zs/jds

��2

� k�k2L1 C 1

4kY k2

S1 C 4

�E�

�Z T

�

�2jzsj2ds

��2

by Condition (11.4). Taking the supremum over all stopping times � 2 T T0 , this

provides:

kY k2S1 C kZk2

H2BMO

� 2k�k2L1 C 1

2kY k2

S1 C 8�22 kzk4

H2BMO

;

and therefore

kY k2S1 C kZk2

H2BMO

� 4k�k2L1 C 16�2

2 kzk4

H2BMO

:

The power 4 on the right-hand side is problematic as it may cause the explosionof the norms, given that the left-hand side is only raised to the power 2! This isprecisely the reason why we need to restrict k�kL1 to be small. For instance, let

R WD 1

16�2

; k�kL1 � R

4and kyk2

S1 C kzk2

H2BMO

� R2:

Then, it follows from the previous estimates that

kY k2S1 C kZk2

H2BMO

� 4R2

16C 16�2

2 R4 D 5R2

16:

Denoting by BR the ball of radius R in S1 � H2BMO, we have then proved that

˚.BR/ � BR:

2. For i D 0; 1 and .yi ; zi / 2 BR, we denote .Y i ; Zi / WD ˚.yi ; zi /, ıy WD y1 �y0,ız WD z1 � z0, ıY WD Y 1 � Y 0, ıZ WD Z1 � Z0, and ıf WD f .z1/ � f .z0/. Weargue as in the previous step: apply Ito’s formula for each stopping time � 2 T T

0 ,take conditional expectations, and maximize over � 2 T T

0 . This leads to

kıY k2S1 C kıZk2

H2BMO

� 16 sup�2T T

0

�E�

�Z T

�

jıfsjds

��2

: (11.5)


We next estimate that

�E�

�Z T

�

jıfsjds

��2

� �22

�E�

�Z T

�

jızs j.jz0s j C jz1

s j/ds

��2

� �22 E�

�Z T

�

jızs j2ds

�E�

�Z T

�

.jz0s j C jz1

s j/2ds

�

� 4R2�22 E�

�Z T

�

jızsj2ds

�:

Then, it follows from (11.5) that

kıY k2S1 C kıZk2

H2BMO

� 16 � 4R2�22 kızk2

H2BMO

� 1

4kızk2

H2BMO

:

Hence, ˚ is a contraction, and there is a unique fixed point. ut

11.2.2 Existence for Bounded Final Condition

We now use the existence result of Theorem 11.6 to build a solution for a quadraticBSDE with general bounded final condition. Let us already observe that, in contrastwith Theorem 11.6, the following construction will only provide existence (and notuniqueness) of a solution .Y; Z/ with bounded Y component. However, this is all weneed to prove in this section as the uniqueness is a consequence of Theorem 11.5.

We first observe that, under Condition (11.2), we may assume without lossof generality that ft .0/ D 0. This is an immediate consequence of the obviousequivalence:

.Y; Z/ solution of BSDE.f; �/ iff . QY ; Z/ solution of BSDE.f; Q�/;

where QYt WD Yt � R t

0fs.0/ds, 0 � t � T , and Q� WD � � R T

0fs.0/ds.

We then continue assuming that ft .0/ D 0.Consider an arbitrary decomposition of the final data � as

� DnX

iD1

�i where k�i kL1 � 1

64�2

: (11.6)

For instance, one may simply take �i WD 1n� and n sufficiently large so that (11.6)

holds true.We will then construct solutions .Y i ; Zi / to quadratic BSDEs with final data �i

as follows:

Step 1. Let f 1 WD f , and define .Y 1; Z1/ as the unique solution of the quadraticBSDE

11.2 Existence 173

Y 1t D �1 C

Z T

t

f 1s .Z1

s /ds �Z T

t

Z1s � dWs; t 2 Œ0; T �: (11.7)

Under Condition (11.2) and Assumption 11.3, there is a unique solution .Y 1; Z1/

with bounded Y 1 and Z1 2 H2BMO. This is achieved by applying Theorem 11.6

under a measure Q defined by the density E.R :

0Dft .Z

0t / � dWt / where Z0 WD 0 and

Dft .0/ is bounded. See also Lemma 11.8.

Step 2. Given .Y j ; Zj /j �i�1, we define the generator

f it .z/ WD ft

�Z

i�1

t C z

� ft

�Z

i�1

t

where Z

i�1

t WDi�1Xj D1

Zjt : (11.8)

We will justify in Lemma 11.8 that there is a unique solution .Y i ; Zi / to the BSDE

Y it D �i C

Z T

t

f is .Zi

s /ds �Z T

t

Zis � dWs; t 2 Œ0; T �; (11.9)

with bounded Y i and such that Zi WD Z1 C � � � C Zi 2 H

2BMO.

Step 3. We finally observe that by setting Y WD Y 1 C � � � C Y n, Z WD Zn, and by

summing the BSDEs (11.9), we directly obtain

Yt DnX

iD1

�i CZ T

t

nXiD1

f is .Zi

s/ds �Z T

t

Zs � dWs

D � CZ T

t

fs.Zs/ds �Z T

t

Zs � dWs;

which means that .Y; Z/ is a solution of our quadratic BSDE of interest. Moreover,Y inherits the boundedness of the Y i ’s, and therefore Z 2 H

2BMO by Lemma 11.2.

Finally, as mentioned before, uniqueness is a consequence of Theorem 11.5.

By the above argument, we have the following existence and uniqueness result.

Theorem 11.7. Let f be a quadratic generator satisfying (11.2) and Assump-tion 11.3. Then, for any � 2 L

1.FT /, there is a unique solution .Y; Z/ 2S1 � H

2BMO to the quadratic BSDE (11.1).

For the proof of this theorem, it only remains to show the existence claim inStep 2.

Lemma 11.8. For i D 1; : : : ; n, let the final data �i be bounded as in (11.6). Thenthere exists a unique solution .Y i ; Zi /1�i�n of the BSDEs (11.9) with bounded Y i ’s.

Moreover, the process Zi WD Z0 C � � � C Zi 2 H

2BMO for all i D 1; : : : ; n.


Proof. We shall argue by induction. That the claim is true for i D 1 was justified inStep 1 above by following exactly the same argument as in Step 2. We next assumethat the claim is true for all j � i � 1 and extend it to i .

1. We first prove a convenient estimate for the generator. Set

�it WD Df i

t .0/ D Dft .Zi�1

/: (11.10)

Then, it follows from the mean value theorem that there exists a random � D�.! 2 Œ0; 1� such that

ˇf i

t .z/ � f it .z0/ � �i

t � .z � z0/ˇ D ˇ

Df it

�z C .1 � �/z0� � Df i

t .0/ˇ jz � z0j

� �2j�z C .1 � �/z0j jz � z0j� �2jz � z0j.jzj C jz0j/: (11.11)

2. We rewrite the BSDE (11.9) into

Y it D �i C

Z T

t

his.Z

is/ds �

Z T

t

Zis � .dWs � �i

s ds/; where his.z/ WD f i

s .z/ � �is � z:

By the definition of the process �i in (11.10), it follows from Assumption 11.3

that j�it j � �1.1 C jZi�1

t j/. Then �i 2 H2BMO is inherited from the induction

hypothesis which guarantees that Zj 2 H2BMO for j � i � 1, and therefore

Zi�1 2 H

2BMO. By Property 3 of BMO martingales, we then conclude that

Bi WD W �Z :

0

�is ds is a Brownian motion under Qi WD E

�Z :

0

�is � dWs

�T

� P:

We now view the latter BSDE as formulated under the equivalent probabilitymeasure Q

i by

Y it D �i C

Z T

t

his.Z

is /ds �

Z T

t

Zis � dBi

s ; Qi � a.s.

where, by (11.11), the quadratic generator hi satisfies the conditions of The-orem 11.6 with the same parameter �2 and the existence of a unique solution.Y i ; Zi/ 2 S1 � H

2BMO.Qi / follows.

3. It remains to prove that Zi WD Z1 C � � � C Zi 2 H

2BMO. To see this, we define

Yi WD Y 1 C� � �CY i , and observe that the pair process .Y

i; Z

i/ solves the BSDE

Yi

t DiX

j D1

�j CZ T

t

iXj D1

f js .Zj

s /ds �Z T

t

Zi

s � dWs

11.3 Portfolio Optimization Under Constraints 175

DiX

j D1

�j CZ T

t

f is .Z

i

s/ds �Z T

t

Zi

s � dWs:

SincePi

j D1 �j is bounded and f i satisfies (11.2), it follows from Lemma 11.2

that Zi 2 H

2BMO. ut

Remark 11.9. The conditions of Assumption 11.3 can be weakened by essentiallyremoving the smoothness conditions. Indeed an existence result was established byKobylansky [26] and Morlais [31] under weaker assumptions.

11.3 Portfolio Optimization Under Constraints

The application of this section was first introduced by Elkaroui and Rouge [17] andImkeller et al. [22].


In this section, we consider a financial market consisting of a non-risky asset,normalized to unity, and d risky assets S D .S1; : : : ; Sd / defined by some initialcondition S0 and the dynamics:

dSt D St ? t .dWt C �tdt/ ;

where � and are bounded progressively measurable processes with values in Rd

and Rd�d , respectively. We also assume that t is invertible with bounded inverse

process �1.In financial words, � is the risk premium process, and is the volatility (matrix)

process.Given a maturity T > 0, a portfolio strategy is a progressively measurable

process f�t ; t � T g with values in Rd and such that

R T

0 j�t j2dt < 1, P�a.s.For each i D 1; : : : ; d and t 2 Œ0; T �, �i

t denotes the dollar amount invested inthe i th risky asset at time t . Then, the liquidation value of a self-financing portfoliodefined by the portfolio strategy � and the initial capital X0 is given by

X�t D X0 C

Z t

0

�r � r .dWr C �r dr/ ; t 2 Œ0; T �: (11.12)

We shall impose more conditions later on the set of portfolio strategies. In particular,we will consider the case where the portfolio strategy is restricted to some

A closed convex subset of Rd :


The objective of the portfolio manager is to maximize the expected utility of thefinal liquidation value of the portfolio, where the utility function is defined by

U.x/ WD �e�x=� for all x 2 R; (11.13)

for some parameter � > 0 representing the risk tolerance of the investor, i.e., ��1 isthe risk aversion.

Definition 11.10. A portfolio strategy � 2 H2loc is said to be admissible if it takes

values in A and

the family˚e�X�

� =�; � 2 T T0

�is uniformly integrable: (11.14)

We denote by A the collection of all admissible portfolio strategies.

We are now ready for the formulation of the portfolio manager problem. Let � besome bounded FT �measurable r.v. representing the liability at the maturity T . Theportfolio manager problem is defined by the stochastic control problem:

V0 WD sup�2A

E�U

X�

T � ��

: (11.15)

Our main objective in the subsequent subsections is to provide a characterization ofthe value function and the solution of this problem in terms of a BSDE.

Remark 11.11. The restriction to the exponential utility case (11.13) is crucial toobtain a connection of this problem to BSDEs.

• In the Markovian framework, we may characterize the value function V bymeans of the corresponding dynamic programming equation. Then, extendingthe definition in a natural way to allow for a changing time origin, the dynamicprogramming equation of this problem is

�@t v � sup�

�� Dxv C 1

2jT�j2Dxxv C .T�/ � .s ? Dxsv/

�D 0:

(11.16)

Notice that the above PDE is fully nonlinear, while BSDEs are connected tosemilinear PDEs. So, in general, there is no reason for the portfolio optimizationproblem to be related to BSDEs.

• Let us continue the discussion of the Markovian framework in the context of anexponential utility. Due to the expression of the liquidation value process (11.13),it follows that U.X

X0;�T / D e�X0=�U.X0;�

T /, where we emphasized the depen-dence of the liquidation value on the initial capital X0. Then, by definition of thevalue function V , we have

V.t; x; s/ D e�x=�V .t; 0; s/;


i.e., the dependence of the value function V in the variable x is perfectlydetermined. By plugging this information into the dynamic programming equa-tion (11.16), it turns out that the resulting PDE for the function U.t; s/ WDV.t; 0; s/ is semilinear, thus explaining the connection to BSDEs.

• A similar argument holds true in the case of power utility function U.x/ D xp=p

for p < 1. In this case, due to the domain restriction of this utility function, onedefines the wealth process X in a multiplicative way, by taking as control Q�t WD�t =Xt , the proportion of wealth invested in the risky assets. Then, it follows thatX

X0; Q�T D X0X

1; Q�T , V.t; x; s/ D xpV.t; 0; s/ and the PDE satisfied by V.t; 0; s/

turns out to be semilinear.

11.3.2 BSDE Characterization

The main result of this section provides a characterization of the portfolio managerproblem in terms of the BSDE:

Yt D � CZ T

t

fr .Zr/dr �Z T

t

Zr � dWr; t � T; (11.17)

where the generator f is given by

ft .z/ WD �z � �t � �

2j�t j2 C 1

2�inf�2A

ˇT

t � � .z C ��t /ˇ2

:

D �z � �t � �

2j�t j2 C 1

2�dist.z C ��t ; t A/2; (11.18)

where for x 2 Rd , dist.x; t A/ denotes the Euclidean distance from x to the set

t A, the image of A by the matrix t .

Example 11.12 (Complete market). Consider the case A D Rd , i.e., no portfolio

constraints. Then ft .z/ D �z � �t � �

2j�t j2 is an affine generator in z, and the above

BSDE can be solved explicitly:

Yt D EQ

t

��

2

Z T

t

j�r j2dr

�; t 2 Œ0; T �;

where Q is the so-called risk-neutral probability measure which turns the processW C R :

0�r dr into a Brownian motion.

Notice that, except for the complete market case A D Rd of the previous

example, the above generator is always quadratic in z. See however Exercise 11.15for another explicitly solvable example.


Since the risk premium process is assumed to be bounded, the above generatorsatisfies Condition (11.2). As for Assumption 11.3, its verification depends on thegeometry of the set A. Finally, the final condition represented by the liability � isassumed to be bounded.

Theorem 11.13. Let A be a closed convex set, and suppose that f satisfiesAssumption 11.3. Then the value function of the portfolio management problem andthe corresponding optimal portfolio are given by

V0 D �e� 1� .X0�Y0/ and O�t WD Arg min

�2AjT

t � � .Zt C ��t /j;

where X0 is the initial capital of the investor and .Y; Z/ is the unique solution ofthe quadratic BSDE (11.17).

Proof. For every � 2 A, we define the process

V �t WD �e�.X

0;�t �Yt /=�; t 2 Œ0; T �:

1. We first compute by Ito’s formula that

dV �t D �1

�V �

t

�dX

0;�t � dYt

C 1

2�2V �

t d hX0;� � Y it

D �1

�V �

t

h.ft .Zt / � 't.Zt ; �t // dt C

Tt �t � Zt

� � dWt

i;

where we denoted:

't .z; �/ WD �Tt � � �t C 1

2�jT

t �t � zj2

D �z � �t � �

2j�t j2 C 1

2�

ˇT

t � � .z C ��t /ˇ2

;

so that ft .z/ D inf�2A 't.z; �/. Consequently, the process V � is a localsupermartingale. Now recall from Theorem 11.7 that the solution .Y; Z/ of thequadratic BSDE has a bounded component Y . Then, it follows from admissibilitycondition (11.14) of Definition 11.10 that the process V � is a supermartingale.In particular, this implies that �e�.X0�Y0/=� � EŒV �

T �, and it then follows fromthe arbitrariness of � 2 A that

V0 � �e�.X0�Y0/=�: (11.19)

2. To prove the reverse inequality, we notice that the portfolio strategy O� introducedin the statement of the theorem satisfies

dV O�t D �1

�V O�

t

T

t O�t � Zt

� � dWt:


Then V O�t is a local martingale. We continue by estimating its diffusion part:

ˇT

t O�t � Zt

ˇ � �j�t j C ˇT

t O�t � .Zt C ��t /ˇ

D �j�t j Cr

ft .Zt / C Zt � �t C �

2j�t j2

� C.1 C jZt j/;

for some constant C . Since Z 2 H2BMO by Theorem 11.7, this implies that T

t O�t �Zt 2 H

2BMO and T

t O�t 2 H2BMO. Then, it follows from Property 3 of BMO

martingales that O� 2 A and V O� is a martingale. Hence,

O� 2 A and E

h�e�.X O�

T �YT /=�i

D �e�.X0�Y0/=�

which, together with (11.19), shows that V0 D �e�.X0�Y0/=� and O� is an optimalportfolio strategy. ut

Remark 11.14. The condition that A is convex in Theorem 11.13 can be droppedby defining the optimal portfolio process O� as a measurable selection in the set ofminimizers of the norm jT

t � � .Zt C ��t /j over � 2 A. See Imkeller et al. [22].

Exercise 11.15. The objective of the following problem is to provide an example ofportfolio optimization problem in incomplete market which can be explicitly solved.This is a non-Markovian version of the PDE based work of Zariphopoulou [42].

1. Portfolio optimization under stochastic volatility Let W D .W 1; W 2/ be astandard Brownian motion on a complete probability space .˝;F ;P/, and denoteF

i WD fF it D .W i

s ; s � t/gt�0, F WD fFt D F1t _ F2

t gt�0.Consider the portfolio optimization problem:

V0 WD sup�2A

E

h� e��.X�

T ��/i;

where � > 0 is the absolute risk-aversion coefficient, � is a boundedFT �measurable random variable, and

X�T WD

Z T

0

�t t

� t dW 1

t Cq

1 � 2t dW 2

t C �t dt

is the liquidation value at time T of a self-financing portfolio � in the financialmarket with zero interest rates and stock price defined by the risk premiumand the volatility processes � and . The latter processes are F�bounded andprogressively measurable. Finally, is a correlation process taking values inŒ0; 1�, and the admissibility set A is defined as in Imkeller-Hu-Muller (see myFields Lecture notes).


2. BSDE characterization. This problem fits in the framework of Hu-Imkeller-Muller of portfolio optimization under constrained portfolio (here the portfoliois constrained to the closed convex subset R � f0g of R2).

We introduce a risk-neutral measure Q under which the process B D.B1; B2/:

B1t WD W 1

t CZ t

0

�r r dr and B2t WD W 2

t CZ t

0

�r

q1 � 2

t dr;

is a Brownian motion.Then, it follows that V0 D e�Y0 , where .Y; Z/ is the unique solution of the

quadratic BSDE:

Yt D � CZ T

t

fr .Zr/dr �Z T

t

Zr � dBr (11.20)

where the generator f W RC � ˝ � R2 is defined by

ft .z/ WD � �2t

2�C �

2

�q1 � 2

t z1 C t z2

2

for all z 2 R2:

The existence of a unique solution to this BSDE with bounded component Y andBMO martingale

R :

0Zt � dBt is guaranteed by our results in the present section.

3. Conditionally Gaussian stochastic volatility. We next specialize the discussionto the case:

� is F1T �measurable, and �; ; are F1�progressively measurable:

Then, by adaptability considerations, it follows that the second component of theZ�process Z2 � 0. Denoting the first component by � WD Z1, this reduces theBSDE (11.20) to

Yt D � CZ T

t

�� 2

t

2�C �

2.1 � 2

t /�2t

dr �

Z T

t

�r dB1r : (11.21)

4. Linearizing the BSDE. To achieve additional simplification, we further assumethat the correlation process is constant. Then for a constant ˇ 2 R, weimmediately compute by Ito’s formula that the process yt WD eˇYt satisfies

dyt

yt

D ˇ

��2

t

2��

2.1 � 2/�2

t

�dt C 1

2ˇ2�2

t dt C ˇ�t dBt

so that the choice

ˇ2 WD �.1 � 2/

11.4 Interacting Investors with Performance Concern 181

leads to a constant generator for y. We now continue in the obvious wayrepresenting y0 as an expected value and deducing Y0.

5. Utility indifference. In the present framework, we may compute explicitly theutility indifference price of the claim �.... This leads to a nonlinear pricing rulewhich has nice financial interpretations.

11.4 Interacting Investors with Performance Concern

11.4.1 The Nash Equilibrium Problem

In this section, we consider N portfolio managers i D 1; : : : ; N whose preferencesare characterized by expected exponential utility functions with tolerance parame-ters �i :

U i .x/ WD �e�x=�i

; x 2 R: (11.22)

In addition, we assume that each investor is concerned about the average perfor-mance of his peers. Given the portfolio strategies �i , i D 1; : : : ; N , of the managers,we introduce the average performance viewed by agent i as

Xi;� WD 1

N � 1

Xj ¤i

X�j

T : (11.23)

The portfolio optimization problem of the i th agent is then defined by

V i0

.�j /j ¤i

� WD V i0

WD sup�i 2Ai

E

hU i

�.1 � �i /X�i

T C �i .X�i

T � Xi;�

T /i

; 1 � i � N;

(11.24)

where �i 2 Œ0; 1� measures the sensitivity of agent i to the performance of his peers,and the set of admissible portfolios Ai is defined as follows.

Definition 11.16. A progressively measurable process �i with values in Rd is said

to be admissible for agent i , and we denote �i 2 Ai if:

1. �i takes values in Ai , a given closed convex subset of Rd .2. EŒ

R T

0j�i

t j2dt � < 1,

3. The familyne�X�i

� =�i; � 2 T

ois uniformly bounded in L

p for all p > 1.

Our main interest is to find a Nash equilibrium, i.e., a situation where all portfoliomanagers are happy with the portfolio given those of their peers.


Definition 11.17. A Nash equilibrium for the N portfolio managers is an N �tuple. O�1; : : : ; O�N / 2 A1 � � � � � AN such that, for every i D 1; : : : ; N , given. O�j /j ¤i , the portfolio strategy O�i is a solution of the portfolio optimization problemV i

0

. O�j /j ¤i

�.

11.4.2 The Individual Optimization Problem

In this section, we provide a formal argument which helps to understand theconstruction of Nash equilibrium of the subsequent section.

For fixed i D 1; : : : ; N , we rewrite (11.24) as

V i0 WD sup

�i 2Ai

E

hU i

�X�i

T � Q�ii

; where Q�i WD �i Xi;�

T : (11.25)

Then, from the example of the previous section, we expect that value function V i0

and the corresponding optimal solution be given by

V i0 D �e�.Xi

0� QY i0 /=�i

; (11.26)

and

Tt O�i

t D ait .

Q�it C �i �t / where ai

t .zi / WD Arg min

ui 2AijT

t ui � zi j; (11.27)

and . QY i ; Q�i / is the solution of the quadratic BSDE:

QY it D Q�i C

Z T

t

�� Q�i

r ��r � �i

2j�r j2C Qf i

r . Q�ir C�i �r /

dr�

Z T

t

Q�ir �dWr; t � T; (11.28)

and the generator Qf i is given by

Qf it .zi / WD 1

2�idist.zi ; t A

i /2; zi 2 Rd : (11.29)

This suggests that one can search for a Nash equilibrium by solving theBSDEs (11.28) for all i D 1; : : : ; N . However, this raises the following difficulties.

The first concern that one would have is that the final data �i does not have to bebounded as it is defined in (11.25) through the performance of the other portfoliomanagers.

But in fact, the situation is even worse because the final data �i induces acoupling of the BSDEs (11.28) for i D 1; : : : ; N . To express this coupling in amore transparent way, we substitute the expressions of �i and rewrite (11.28) fort D 0 into:


QY i0 D �i � C

Z T

0

Qf ir .�i

r /dr �Z T

0

0@�i

r � �iN

Xj ¤i

ajr .�j

r /

1A � dBr

where the process B WD W C R :

0�r dr is the Brownian motion under the equivalent

martingale measure,

�iN WD �i

N � 1; �i

t WD Q�it C �i �t ; t 2 Œ0; T �;

and the final data is expressed in terms of the unbounded r.v.

� WDZ T

0

�r � dBr � 1

2

Z T

0

j�t j2dt:

Then QY0 D Y0, where .Y; �/ is defined by the BSDE

Y it D �i � C

Z T

t

Qf ir .�i

r /dr �Z T

t

0@�i

r � �iN

Xj ¤i

ajr .�j

r /

1A � dBr: (11.30)

In order to sketch (11.30) into the BSDEs framework, we further introduce themapping �t W RNd �! R

Nd defined by the components:

�it .�

1; : : : ; �N / WD �i � �iN

Xj ¤i

ajt .�j / for all �1; : : : ; �N 2 R

d : (11.31)

It turns out that the mapping �t is invertible under fairly general conditions. We shallprove this result in Lemma 11.18 in the case where the Ai ’s are linear subspaces ofR

d . Then one can rewrite (11.30) as

Y it D �i � C

Z T

t

f ir .Zr /dr �

Z T

t

Zir � dBr; (11.32)

where the generator f i is now given by

f i .z/ WD Qf ir

f��1t .z/gi

�for all z D .z1; : : : ; zN / 2 R

Nd ; (11.33)

and f��1t .z/gi indicates the i th block component of size d of ��1

t .z/.

11.4.3 The Case of Linear Constraints

We now focus on the case where the constraints sets are such that

Ai is a linear subspace of Rd ; i D 1; : : : ; N: (11.34)


Then, denoting by P it the orthogonal projection operator on t A

i (i.e., the image ofAi by the matrix t ), we immediately compute that

ait .�

i / WD P it .�i / (11.35)

and

�it .�

1; : : : ; �N / WD �i � �iN

Xj ¤i

Pjt .�j /; for i D 1; : : : ; N: (11.36)

Lemma 11.18. Let .Ai /1�i�N be linear subspaces of Rd . Then, for all t 2

Œ0; T �:

(i) The linear mapping �t of (11.36) is invertible if and only if

NYiD1

�i < 1 orN\

iD1

Ai D f0g: (11.37)

(ii) This condition is equivalent to the invertibility of the matrices Id � Qit ,

i D 1; : : : ; N , where

Qit WD

Xj ¤i

�jN

1 C �jN

Pjt .Id C �

jN P i

t /:

(iii) Under (11.37), the i th component of ��1t is given by

f��1t .z/gi D .Id � Qi

t /�1

0@zi C

Xj ¤i

1

1 C �jN

Pjt .�i

N zj � �jN zi /

1A :

Proof. We omit all t subscripts, and we denote �i WD �iN . For arbitrary z1; : : : ; zN

in Rd , we want to find a unique solution to the system

�i � �iXj ¤i

P j �j D zi ; 1 � i � N: (11.38)

1. Since P j is a projection, we immediately compute that .Id C �j P j /�1 D Id ��j

1C�j P j . Subtracting equations i and j from the above system, we see that

�i P j �j D P j .Id C �j P j /�1��j .Id C �i P i /�i C �i zj � �j zi

D 1

1 C �jP j

��j .Id C �i P i /�i C �i zj � �j zi

:


Then it follows from (11.38) that

zi D �i �Xj ¤i

1

1 C �jP j

��j .Id C �i P i /�i C �i zj � �j zi

;

and we can rewrite (11.38) equivalently as

0@Id �

Xj ¤i

�j

1 C �jP j .Id C �i P i /

1A �i D zi C

Xj ¤i

1

1 C �jP j .�i zj � �j zi /;

(11.39)

so that the invertibility of � is now equivalent to the invertibility of the matricesId � Qi , i D 1; : : : ; N , where Qi is introduced in statement of the lemma.

2. We now prove that Id � Qi is invertible for every i D 1; : : : ; N iff (11.37) holdstrue.

(a) First, assume to the contrary that �i D 1 for all i and \NiD1A

i contains anonzero element x0. Then, it follows that y0 WD Tx0 satisfies P i y0 D y0

for all i D 1; : : : ; N , and therefore Qiy0 D y0. Hence, Id � Qi is notinvertible.

(b) Conversely, we consider separately two cases.

• If �i0 < 1 for some i0 2 f1; : : : ; N g, we estimate that

�i0

1 C �i0<

1N �1

1 C 1N �1

and�i

1 C �i�

1N �1

1 C 1N �1

for i ¤ i0:

Then for all i ¤ i0 and x ¤ 0, it follows that jQixj < jxj proving thatI � Qi is invertible.

• If �i D 1 for all i D 1; : : : ; N , then for all x 2 Ker.Qi/, we havex D Qix and therefore

jxj Dˇˇ X

j ¤i

�j

1 C �jP j .Id C �i P i /x

ˇˇ

D 1

N

ˇˇ X

j ¤i

P j .Id C 1

N � 1P i /x

ˇˇ

� 1

N

Xj ¤i

.1 C 1

N � 1jxj D jxj;

where we used the fact that the spectrum of the P i ’s is reduced to f0; 1g.Then equality holds in the above inequalities, which can only happen if


P i x D x for all i D 1; : : : ; N . We can then conclude that \NiD1Ker.Id �

P i / D f0g implies that Id � Qi is invertible. This completes the proof as\N

iD1Ker.Id � P i / D f0g is equivalent to \NiD1A

i D f0g. ut

11.4.4 Nash Equilibrium Under Deterministic Coefficients

The discussion of Sect. 11.4.2 shows that the question of finding a Nash equilibriumfor our problem reduces to the vector BSDE with quadratic generator (11.32), thatwe rewrite here for convenience:

Y it D �i � C

Z T

t

f ir .Zr /dr �

Z T

t

Zir � dBr; (11.40)

where � WD R T

0�r � dBr � 1

2

R T

0j�r j2dr , and the generator f i is given by:

f i.z/ WD Qf ir

f��1t .z/gi

�for all z D .z1; : : : ; zN / 2 R

Nd : (11.41)

Unfortunately, the problem of solving vector BSDEs with quadratic generator isstill not understood. Therefore, we will not continue in the generality assumed sofar, and we will focus in the sequel on the case where

the Ai ’s are vector subspaces of Rd and

t D .t/ and �t D �.t/ are deterministic functions. (11.42)

Then, the vector BSDE reduces to

Y it D �i � C 1

2�i

Z T

t

ˇˇ.Id � P i .t//

f�.t/�1.Zr /gi�ˇˇ2

dr �Z T

t

Zir � dBr;

(11.43)

where P it D P i .t/ is deterministic and f��1

t .z/gi D f�.t/�1.z/gi is deterministicand given explicitly by Lemma 11.18(iii).

In this case, an explicit solution of the vector BSDE is given by

Zit D �i �.t/

Y it D ��i

2

Z T

0

j�.t/j2dt C 1

2�i

Z t

0

j.Id � P i .t//M i.t/�.t/j2dt; (11.44)


where

M i.t/ WD0@Id �

Xj ¤i

�jN

1 C �jN

P j .t/.Id C �jN P i .t/

1A

�1

�0@�i Id C

Xj ¤i

1

1 C �jN

P j .t/.�iN �j � �

jN �i /

1A :

By (11.27), the candidate for agent i th optimal portfolio is also deterministic andgiven by

O�i WD �1P i M i�; i D 1; : : : ; N: (11.45)

Proposition 11.19. In the context of the financial market with deterministic coeffi-cients (11.42), the N �tuple . O�1; : : : ; O�N / defined by (11.45) is a Nash equilibrium.

Proof. The above explicit solution of the vector BSDE induces an explicit solution. QY i ; Q�i / of the coupled system of BSDEs (11.28), 1 � i � N with deterministic Q�i .In order to prove the required result, we have to argue by verification following thelines of the proof of Theorem 11.13 for every fixed i in f1; : : : ; ng.

1. First for an arbitrary �i , we define the process

V �i

t WD �e�.X�it � QY i

t /=�i

; t 2 Œ0; T �:

By Ito’s formula, it is immediately seen that this process is a local supermartin-gale (the generator has been defined precisely to satisfy this property!). By theadmissibility condition of Definition 11.16 together with the fact that QY i has aGaussian distribution (as a diffusion process with deterministic coefficients), itfollows that the family fV �i

� ; � 2 T g is uniformly bounded in L1C" for any " > 0.

Then the process V �iis a supermartingale. By the arbitrariness of �i 2 Ai , this

provides the first inequality

�e�.Xi0� QY i

0 /=�i � V i0

. O�j /j ¤i

�:

2. We next prove that equality holds by verifying that O�i 2 Ai and the process V O�i

is a martingale. This will provide the value function of agent i ’s portfolio opti-mization problem and the fact that O�i is optimal for the problem V i

0

. O�j /j ¤i

�.

That O�i 2 Ai is immediate; recall again that O�i is deterministic. As inthe previous step, direct application of Ito’s formula shows that V O�i

is a localmartingale, and the martingale property follows from the fact that X O�i

and QY i havedeterministic coefficients. ut

We conclude this section with a simple example which shows the effect of theinteraction between managers.


Example 11.20 (N D 3 investors, d D 3 assets). Consider a financial market withN D d D 3. Denoting by .e1; e2; e3/ the canonical basis of R3, the constraints setfor the agents are

A1 D Re1 C Re2; A2 D Re2 C Re3; A3 D Re3;

i.e., agent 1 is allowed to trade without constraints the first two assets, agent 2 isallowed to trade without constraints the last two assets, and agent 3 is only allowedto trade the third assets without constraints.

We take D I3. In the present context of deterministic coefficients, this meansthat the price processes of the assets are independent. Therefore, if there were nointeraction between the investors, their optimal investment strategies would not beaffected by the assets that they are not allowed to trade.

In this simple example, all calculations can be performed explicitly. The Nashequilibrium of Proposition 11.19 is given by

O�1t D ��1.t/e1 C 2 C �1

2 � �1�2

2

��2.t/e2;

O�2t D 2 C �2

2 � �1�2

2

��2.t/e2 C 2 C �2

2 � �2�3

2

��3.t/e3; and

O�3t D 2 C �3

2 � �2�3

2

��3.t/e3:

This shows that, whenever two investors have access to the same asset, theirinteraction induces an overinvestment in this asset characterized by a dilation factorrelated to the their sensitivity to the performance of the other investor.

Chapter 12Probabilistic Numerical Methodsfor Nonlinear PDEs

In this chapter, we introduce a backward probabilistic scheme for the numericalapproximation of the solution of a nonlinear partial differential equation. Thescheme is decomposed into two steps:

1. The Monte Carlo step consists in isolating the linear generator of some underly-ing diffusion process, so as to split the PDE into this linear part and a remainingnonlinear one.

2. Evaluating the PDE along the underlying diffusion process, we obtain a naturaldiscrete-time approximation by using finite differences approximation in theremaining nonlinear part of the equation.

Our main concern will be to prove the convergence of this discrete-time approx-imation. In particular, the above scheme involves the calculation of conditionalexpectations that should be replaced by some approximation for any practicalimplementation. The error analysis of this approximation will not be addressed here.

Throughout this chapter,� and � are two functions from RC �Rd to R

d and Sd ,respectively. Let a WD �2, and define the linear operator:

LX' WD @'

@tC � �D' C 1

2a �D2':

Consider the map

F W .t; x; r; p; �/ 2 RC � Rd � R � R

d � Sd 7�! F.x; r; p; �/ 2 R;

which is assumed to be elliptic:

F.t; x; r; p; �/ � F.t; x; r; p; � 0/ for all � � � 0:

Our main interest is on the numerical approximation for the Cauchy problem:

�LXv � F ��; v;Dv;D2v� D 0; on Œ0; T / � R

d ; (12.1)

v.T; �/ D g; on 2 Rd : (12.2)


189

190 12 Probabilistic Numerical Methods for Nonlinear PDEs

12.1 Discretization

Let W be an Rd -valued Brownian motion on a filtered probability space

.�;F ;F;P/.For a positive integer n, let h WD T=n, ti D ih, i D 0; : : : ; n, and consider the

one step ahead Euler discretization

OXt;xh WD x C �.t; x/hC �.t; x/.WtCh �Wt/; (12.3)

of the diffusion X corresponding to the linear operator LX . Our analysis doesnot require any existence and uniqueness result for the underlying diffusion X .However, the subsequent formal discussion assumes it in order to provide a naturaljustification of our numerical scheme.

Assuming that the PDE (12.1) has a classical solution, it follows from Ito’sformula that

Eti ;x

�v

�tiC1; XtiC1

�� D v .ti ; x/C Eti ;x

�Z tiC1

ti

LXv.t; Xt/dt

�

where we ignored the difficulties related to the local martingale part, and Eti ;x WDEŒ�jXti D x� denotes the expectation operator conditional on fXti D xg. Since vsolves the PDE (12.1), this provides

v.ti ; x/ D Eti ;x

�v

�tiC1; XtiC1

�� C Eti ;x

�Z tiC1

ti

F .�; v;Dv;D2v/.t; Xt/dt

�:

By approximating the Riemann integral, and replacing the process X by its Eulerdiscretization, this suggests the following approximation:

vh.T; :/ WD g and vh.ti ; x/ WD Rti Œvh.tiC1; :/�.x/; (12.4)

where we denoted for a function W Rd �! R with exponential growth:

Rt Œ �.x/ WD E

h . OXt;x

h /i

C hF .t; �;Dh / .x/; (12.5)

with Dh WD �D0h ;D1

h ;D2h

�T, and

Dkh .x/ WD E

hDk . OXt;x

h /i

for k D 0; 1; 2;

and Dk is the kth order partial differential operator with respect to the spacevariable x. The differentiations in the above scheme are to be understood in the senseof distributions. This algorithm is well defined whenever g has exponential growthand F is a Lipschitz map. To see this, observe that any function with exponential

12.1 Discretization 191

growth has weak gradient and Hessian and the exponential growth is inherited ateach time step from the Lipschitz property of F .

At this stage, the above backward algorithm presents the serious drawbackof involving the gradient Dvh.tiC1; :/ and the Hessian D2vh.tiC1; :/ in order tocompute vh.ti ; :/. The following result avoids this difficulty by an easy integration-by-parts argument.

Lemma 12.1. Let f W Rd ! R be a function with exponential growth. Then

E

hDif . OXti ;x

h /i

D E

hf . OXti ;x

h /Hhi .ti ; x/

ifor i D 1; 2;

where

Hh1 D 1

h��1Wh and Hh

2 D 1

h2��1�WhW

Th � hId

��1: (12.6)

Proof. We only provide the argument in the one-dimensional case; the extensionto any dimension d is immediate. Let G be a one-dimensional Gaussian randomvariable with men m and variance v. Then, for any function f with exponentialgrowth, it follows from an integration by parts that

EŒf 0.G/� DZf 0.s/e� 1

2.s�m/2

vdsp2�v

DZf .s/

s �mv

e� 12.s�m/2

vdsp2�v

D E

�f .G/

G �m

v

�;

where the remaining term in the integration-by-parts formula vanishes by theexponential growth of f . This implies the required result for i D 1.

To obtain the result for i D 2, we continue by integrating by parts once more:

EŒf 00.G/� D E

�f 0.G/

G �mv

�

DZf 0.s/

s �mv

e� 12.s�m/2

vdsp2�v

DZf .s/

��1

vC

s �mv

2�e� 1

2.s�m/2

vdsp2�v

D E

�f .G/

.G �m/2 � v

v2

�: ut

In the sequel, we shall denote Hh WD .1;Hh1 ;H

h2 /

T. In view of the last lemma,we may rewrite the discretization scheme (12.4) into

vh.T; :/ D g and vh.ti ; x/ D Rti

�vh.tiC1; :/

�.x/; (12.7)


where

Rti Œ �.x/ D E

h . OXt;x

h /i

C hF .t; �;Dh / .x/

and

Dkh .x/ WD E

h . OXt;x

h /Hkh .t; x/

ifor k D 0; 1; 2: (12.8)

Observe that the choice of the drift and the diffusion coefficients � and � in thenonlinear PDE (12.1) is arbitrary. So far, it has been only used in order to define theunderlying diffusionX . Our convergence result will however place some restrictionson the choice of the diffusion coefficient, see Remark 12.6.

Once the linear operator LX is chosen in the nonlinear PDE, the above algorithmhandles the remaining nonlinearity by the classical finite differences approximation.This connection with finite differences is motivated by the following formalinterpretation of Lemma 12.1, where, for ease of presentation, we set d D 1,� � 0,and �.x/ � 1:

• Consider the binomial random walk approximation of the Brownian motionOWtk WD Pk

jD1 wj , tk WD kh; k � 1, where fwj ; j � 1g are independent random

variables distributed as 12

ıfp

hg C ıf�phg

. Then, this induces the following

approximation:

D1h .x/ WD E

� .X

t;xh /Hh

1

� � .x C ph/ � .x � p

h/

2ph

;

which is the centered finite differences approximation of the gradient.• Similarly, consider the trinomial random walk approximation OWtk WD Pk

jD1 wj ,tk WD kh; k � 1, where fwj ; j � 1g are independent random variables distributed

as 16

ıfp

3hg C 4ıf0g C ıf�p3hg

, so that EŒwnj � D EŒW n

h � for all integers n � 4.

Then, this induces the following approximation:

D2h .x/ WD E

� .Xt;x

h /Hh2

�

� .x C p3h/ � 2 .x/C .x � p

3h/

3h;

which is the centered finite differences approximation of the Hessian.

In view of the above interpretation, the numerical scheme (12.7) can be viewed asa mixed Monte Carlo–Finite Differences algorithm. The Monte Carlo componentof the scheme consists in the choice of an underlying diffusion process X . Thefinite differences component of the scheme consists in approximating the remainingnonlinearity by means of the integration-by-parts formula of Lemma 12.1.

12.2 Convergence of the Discrete-Time Approximation 193

12.2 Convergence of the Discrete-Time Approximation

The main convergence result of this section requires the following assumptions:

Assumption 12.2. The PDE (12.1) has comparison for bounded functions, i.e., forany bounded upper-semicontinuous viscosity subsolution u and any bounded lower-semicontinuous viscosity supersolution v on Œ0; T / � R

d , satisfying

u.T; �/ � v.T; �/;we have u � v.

For our next assumption, we denote by Fr , Fp , and F� the partial gradients of Fwith respect to r , p, and � , respectively. We also denote by F�

� the pseudo-inverseof the nonnegative symmetric matrix F� . We recall that any Lipschitz function isdifferentiable a.e.

Assumption 12.3. (i) The nonlinearity F is Lipschitz continuous with respect to.x; r; p; �/ uniformly in t , and jF.�; �; 0; 0; 0/j1 < 1.

(ii) F is elliptic and dominated by the diffusion of the linear operator LX , i.e.

F� � a on Rd � R � R

d � Sd : (12.9)

(iii) Fp 2 Image.F�/ andˇˇF Tp F

�� Fp

ˇˇ1 < C1.

Before commenting this assumption, we state our main convergence result.

Theorem 12.4. Let Assumptions 12.2 and 12.3 hold true, and assume that �; � areLipschitz-continuous and � is invertible. Then for every bounded Lipschitz functiong, there exists a bounded function v so that

vh �! v locally uniformly:

In addition, v is the unique bounded viscosity solution of problem (12.1) and (12.2).

The proof of this result is reported in the Sect. 12.4. We conclude by someremarks.

Remark 12.5. Assumption 12.3(iii) is equivalent to

jm�F j1 < 1 where mF WD min

w2Rd˚Fp � w C wTF�w

�: (12.10)

To see this observe first that F� is a symmetric matrix, as a consequence of theellipticity of F . Then, any w 2 R

d has an orthogonal decomposition w D w1Cw2 2Ker.F� /˚ Image.F� /, and by the nonnegativity of F� ,

Fp � w C wTF�w D Fp � w1 C Fp � w2 C wT2F�w2

D �14F Tp F

�� Fp C Fp � w1 C

ˇˇˇ1

2.F�

� /1=2 � Fp � F 1=2

� w2

ˇˇˇ

2

:


Remark 12.6. Assumption 12.3(ii) places some restrictions on the choice of thelinear operator LX in the nonlinear PDE (12.1). First, F is required to be uniformlyelliptic, implying an upper bound on the choice of the diffusion matrix � . Since�2 2 SC

d , this implies in particular that our main results do not apply to generaldegenerate nonlinear parabolic PDEs. Second, the diffusion of the linear operator �is required to dominate the nonlinearity F which places implicitly a lower bound onthe choice of the diffusion � .

Example 12.7. Let us consider the nonlinear PDE in the one-dimensional case� @v@t

� 12

�a2vC

xx � b2v�xx

�where 0 < b < a are given constants. Then if we restrict

the choice of the diffusion to be constant, it follows from Assumption 12.3 that13a2 � �2 � b2, which implies that a2 � 3b2. If the parameters a and b do not

satisfy the latter condition, then the diffusion � has to be chosen to be state and timedependent.

Remark 12.8. Under the boundedness condition on the coefficients � and � , therestriction to a bounded terminal data g in Theorem 12.4 can be relaxed by animmediate change of variable. Let g be a function with ˛�exponential growth forsome ˛ > 0. Fix some M > 0, and let � be an arbitrary smooth positive functionwith

�.x/ D e˛jxj for jxj � M;

so that both �.x/�1r�.x/ and �.x/�1r2�.x/ are bounded. Let

u.t; x/ WD �.x/�1v.t; x/ for .t; x/ 2 Œ0; T � � Rd :

Then, the nonlinear PDE problem (12.1) and (12.2) satisfied by v converts into thefollowing nonlinear PDE for u:

�LXu � QF ��; u;Du;D2u� D 0 on Œ0; T / � R

d

v.T; �/ D Qg WD ��1g on Rd ; (12.11)

where

QF .t; x; r; p; �/ WD r�.x/ � ��1r�C 1

2Tr

�a.x/

�r��1r2�C 2p��1r�T��

C��1F�t; x; r�; rr�C p�; rr2�C 2pr�T C ��

�:

Recall that the coefficients � and � are assumed to be bounded. Then, it is easy tosee that QF satisfies the same conditions as F . Since Qg is bounded, the convergenceTheorem 12.4 applies to the nonlinear PDE (12.11).

12.3 Consistency, Monotonicity and Stability 195

12.3 Consistency, Monotonicity and Stability

The proof of Theorem 12.4 is based on the monotone schemes method of Barles andSouganidis [4] which exploits the stability properties of viscosity solutions. Themonotone schemes method requires three conditions: consistency, monotonicity,and stability that we now state in the context of backward scheme (12.7).

To emphasize on the dependence on the small parameter h in this section, wewill use the notation

ThŒ'�.t; x/ WD Rt Œ'.t C h; :/�.x/ for all ' W RC � Rd �! R:

Lemma 12.9 (Consistency). Let ' be a smooth function with bounded derivatives.Then for all .t; x/ 2 Œ0; T � � R

d

lim.t0 ; x0/ ! .t; x/

.h; c/ ! .0; 0/

t0 C h � T

�Œc C '�� ThŒc C '�

�.t 0; x0/

hD � �LX' C F.�; ';D';D2'/

�.t; x/:

The proof is a straightforward application of Ito’s formula, and is omitted.

Lemma 12.10 (Monotonicity). Let '; W Œ0; T � � Rd �! R be two Lipschitz

functions with ' � . Then

ThŒ'�.t; x/ � ThŒ �.t; x/C Ch E

h. � '/.t C h; OXt;x

h /i

for some C > 0

depending only on the constantmF in (12.10).

Proof. By Lemma 12.1, the operator Th can be written as

ThŒ �.t; x/ D E

h . OXt;x

h /i

C hFt; x;E

h . OXt;x

h /Hh.t; x/i:

Let f WD � ' � 0 where ' and are as in the statement of the lemma. Let Fdenote the partial gradient with respect to D .r; p; �/. By the mean value theorem

ThŒ �.t; x/ � ThŒ'�.t; x/ D E

hf . OXt;x

h /i

C hF ./ � Dhf . OXt;xh /

D E

hf . OXt;x

h / .1C hF ./ �Hh.t; x//i;

for some D .t; x; Nr; Np; N�/. By the definition ofHh.t; x/

ThŒ � � ThŒ'�

D E

hf . OXt;x

h /�1C hFr C Fp:�

�1Wh C h�1F� � ��1.WhWTh � hI /��1�i ;


where the dependence on and x has been omitted for notational simplicity. SinceF� � a by Assumption 12.3, we have 1 � a�1 � F� � 0 and therefore

ThŒ � � ThŒ'� � E

hf . OXt;x

h /�hFr C Fp:�

�1Wh C h�1F� � ��1WhWTh �

�1�i

D E

�f . OXt;x

h /

�hFr C hFp:�

�1 Wh

hC hF� � ��1 WhW

Th

h2��1

��:

Recall the functionmF defined in (12.10). Under Assumption 12.3, it follows fromRemark 12.5 that K WD jm�

F j1 < 1. Then

Fp:��1 Wh

hC hF� � ��1 WhW

Th

h2��1 � �K;

and therefore:

ThŒ � � ThŒ'� � E

hf . OXt;x

h / .hFr � hK/i

� �C 0hEhf . OXt;x

h /i

for some constant C > 0, where the last inequality follows from (12.10). utLemma 12.11 (Stability). Let '; W Œ0; T � � R

d �! R be two L1�bounded

functions. Then there exists a constant C > 0 such that

ˇˇThŒ'� � ThŒ �

ˇˇ1 � j' � j1.1C Ch/:

In particular, if g is L1�bounded, then the family .vh/h defined in (12.7) is

L1�bounded, uniformly in h.

Proof. Let f WD ' � . Arguing as in the previous proof, we see that

ThŒ'� � ThŒ � D E

�f . OXh/

�1 � a�1 � F� C hjAhj2 C hFr � h

4F Tp F

�� Fp

��:

where

Ah D 1

2.F�

� /1=2Fp � F 1=2

� ��1 Wh

h:

Since 1�TrŒa�1F� � � 0, jFr j1 < 1, and jF Tp F

�� Fpj1 < 1 by Assumption 12.3,

it follows that

jThŒ'� � ThŒ �j1 � jf j1�1 � a�1 � F� C hEŒjAhj2�C Ch

�:

But, EŒjAhj2� D h4F Tp F

�� Fp C a�1 � F� . Therefore, using again Assumption 12.3,

we see that

12.4 The Barles–Souganidis Monotone Scheme 197

jThŒ'� � ThŒ �j1 � jf j1�1C h

4F Tp F

�� Fp C Ch

�� jf j1.1C NCh/:

To prove that the family .vh/h is bounded, we proceed by backward induction. Bythe assumption of the lemma vh.T; :/ D g is L1�bounded. We next fix some i < nand we assume that jvh.tj ; :/j1 � Cj for every i C 1 � j � n � 1. Proceeding asin the proof of Lemma 12.10 with ' � vh.tiC1; :/ and � 0, we see that

ˇˇvh.ti ; :/

ˇˇ1 � h jF.t; x; 0; 0; 0/j C CiC1.1C Ch/:

Since F.t; x; 0; 0; 0/ is bounded by Assumption 12.3, it follows from the discreteGronwall inequality that jvh.ti ; :/j1 � C eCT for some constant C independentof h. ut

12.4 The Barles–Souganidis Monotone Scheme

This section is dedicated to the proof of Theorem 12.4. We emphasize on the factthat the subsequent argument applies to any numerical scheme which satisfies theconsistency, monotonicity, and stability properties. In the present situation, we alsoneed to prove a technical result concerning the limiting behavior of the boundarycondition at T . This will be needed in order to use the comparison result whichis assumed to hold for the equation. The statement and its proof are collected inLemma 12.12.

Proof of Theorem 12.4. 1. By the stability property of Lemma 12.11, it followsthat the relaxed semicontinuous envelopes

v.t; x/ WD lim inf.h;t 0;x0/!.0;t;x/

vh.t 0; x0/ and v.t; x/ WD lim sup.h;t 0;x0/!.0;t;x/

vh.t 0; x0/

are bounded. We shall prove in Step 2 that v and v are viscosity supersolutionand subsolution, respectively. The final ingredient is reported in Lemma 12.12which states that v.T; :/ D v.T; :/. Then, the proof is completed by appealing tothe comparison property of Assumption 12.2.

2. We only prove that v is a viscosity supersolution of (12.1). The proof of theviscosity subsolution property of v follows exactly the same line of argument.Let .t0; x0/ 2 Œ0; T / � R

d and ' 2 C2�Œ0; T � � R

d�

be such that

0 D .v � '/.t0; x0/ D (strict) minŒ0;T ��Rd

.v � '/: (12.12)

Since vh is uniformly bounded in h, we may assume without loss of generalitythat ' is bounded. Let .hn; tn; xn/n be a sequence such that


hn ! 0; .tn; xn/ ! .t0; x0/; and vhn.tn; xn/ �! v.t0; x0/: (12.13)

For a positive scalar r with 2r < T � t0, we denote by Br.tn; xn/ the ball ofradius r centered at .tn; xn/, and we introduce

ın WD .vhn� � '/.Otn; Oxn/ D minBr.tn;xn/

.vhn� � '/; (12.14)

where vhn� is the lower-semicontinuous envelope of vhn . We claim that

ın �! 0 and .Otn; Oxn/ �! .t0; x0/: (12.15)

This claim is proved in Step 3 . By the definition of vhn� , we may find a sequence.Ot 0n; Ox0

n/n�1 converging to .t0; x0/, such that

jvhn.Ot 0n; Ox0n/ � vhn� .Otn; Oxn/j � h2n and j'.Ot 0n; Ox0

n/� '.Otn; Oxn/j � h2n: (12.16)

By (12.14), (12.16), and the definition of the functions vh in (12.7), we have

2h2n C ın C '.Ot 0n; Ox0n/ � h2n C ın C '.Otn; Oxn/

D h2n C vhn� .Otn; Oxn/� vhn.Ot 0n; Ox0

n/

D Thn Œvhn �.Ot 0n; Ox0

n/

� Thn Œ'hn C ın�.Ot 0n; Ox0

n/

CChnEh.vhn � ' � ın/

OX Ot 0n; Ox0n

hn

i;

where the last inequality follows from (12.14) and the monotonicity property ofLemma 12.10. Dividing by hn, the extremes of this inequality provide

ın C '.Ot 0n; Ox0n/� Thn Œ'

hn C ın�.Ot 0n; Ox0n/

hn� CE

h.uhn � ' � ın/

OX Ot 0n; Ox0n

hn

i:

We now send n to infinity. The right hand side converges to zeroby (12.13), (12.15), and the dominated convergence theorem. For the left hand-side term, we use the consistency result of Lemma 12.9. This leads to

� � LX' � F.:; ';D';D2'/�.t0; x0/ � 0;

as required.3. We now prove Claim (12.15). Since .Otn; Oxn/n is a bounded sequence, we may

extract a subsequence, still named .Otn; Oxn/n, converging to some .Ot ; Ox/. Then:

12.4 The Barles–Souganidis Monotone Scheme 199

0 D .v � '/.t0; x0/

D limn!1.v

hn � '/.tn; xn/

� lim supn!1

.vhn� � '/.tn; xn/

� lim supn!1

.vhn� � '/.Otn; Oxn/

� lim infn!1 .vhn� � '/.Otn; Oxn/

� .v � '/.Ot ; Ox/:

Since .t0; x0/ is a strict minimizer of the difference .v � '/, this implies (12.15).ut

The following result is needed in order to use the comparison property ofAssumption 12.2. We shall not report its long technical proof, see [19].

Lemma 12.12. The function vh is Lipschitiz in x, 1=2�Holder continuous in t ,uniformly in h, and for all x 2 R

d , we have

jvh.t; x/ � g.x/j � C.T � t/ 12 :

Chapter 13Introduction to Finite Differences Methods

by Agnes Tourin�

In this lecture, I discuss the practical aspects of designing Finite Difference methodsfor Hamilton–Jacobi–Bellman equations of parabolic type arising in quantitativefinance. The approach is based on the very powerful and simple frameworkdeveloped by Barles– Souganidis [4], see the review of the previous chapter. Thekey property here is the monotonicity which guarantees that the scheme satisfies thesame ellipticity condition as the HJB operator. I will provide a number of examplesof monotone schemes in these notes. In practice, pure finite difference schemes areonly useful in 1, 2, or at most 3 spatial dimensions. One of their merits is to be quitesimple and easy to implement. Also, as shown in the previous chapter, they can alsobe combined with Monte Carlo methods to solve nonlinear parabolic PDEs.

Such approximations are now fairly standard, and you will find many interestingexamples available in the literature. For instance, I suggest the articles on the subjectby Forsyth (see [33, 41, 43]). There is also a classical book written by Kushner andPaul Dupuis [28] on numerical methods for stochastic control problems. Finally,for a basic introduction to finite difference methods for linear parabolic PDEs, Irecommend the book by Thomas [40].

13.1 Overview of the Barles–Souganidis Framework

Consider the parabolic PDE

ut � F.t; x; u;Du;D2u/ D 0 in .0; T � � RN ; (13.1)

u.0; x/ D u0.x/ in RN ; (13.2)

�Fields Institute Research Immersion Fellow, 2010


201

202 13 Introduction to Finite Differences Methods

where F is elliptic

F.t; x; u; p; A/ � F.t; x; u; p; B/; if A � B:

For the sake of simplicity, we assume that u0 is bounded in RN .

The main application we have in mind is, for instance, to an operator F comingfrom a stochastic control problem:

F.t; x; r; p;X/ D sup˛2A

f�TrŒa˛.t; x/X� � b˛.t; x/p � c˛.t; x/r � f ˛.t; x/g;

where a˛ D 12�˛�˛T .

Typically, the set of control A is compact or finite, all the coefficients in theequations are bounded and Lipschitz continuous in x, Holder with coefficient 1

2in

t and all the bounds are independent of ˛. Then the unique viscosity solution u of(13.1) is a bounded and Lipschitz continuous function and is the solution of theunderlying stochastic control problem. The ideas, concepts, and techniques actuallyapply to a broader range of optimal control problems. In particular, you can adapt thetechniques to handle different situations, even possibly treat some delicate singularcontrol problems.

In the previous chapter, our convergence result required the technical,Lemma 12.12 in order for the comparison result to apply. However, an easierstatement of the Barles–Souganidis method can be obtained at the price of assuminga stronger comparison result in the following sense.

Definition 13.1. We say that the problem (13.1) and (13.2) satisfies the strongcomparison principle for bounded solution if for all bounded functions u 2 USCand v 2 LSC such that

• u (resp. v) is a viscosity subsolution (resp. supersolution) of (13.1) on .0; T ��RN .

• The boundary condition holds in the viscosity sense

maxfut � F.:; u;Du;D2u/; u � u0g � 0 on f0g � RN

minfut � F.:; u;Du;D2u/; u � u0g � 0 on f0g � RN I

we have u � v on Œ0; T � � RN .

Under the strong comparison principle, any monotonic stable and consistentscheme achieves convergence, and there is no need to analyze the behavior of thescheme near the boundary.

The aim is to build an approximation scheme which preserves the ellipticity. Thisdiscrete ellipticity property is called monotonicity. The monotonicity, together withthe consistency of the scheme and some regularity, ensures its convergence to theunique viscosity solution of the PDE (13.1), (13.2). It is worth insisting on the factthat if the scheme is not monotone, it may fail to converge to the correct solution(see [33] for an example)! We present the theory rather informally, and we refer tothe original articles for more details. The general concepts and machinery apply to a

13.2 First Examples 203

wide range of equations, but the reader needs to be aware that each PDE has its ownpeculiarities and that, in practice, the techniques must be tailored to each particularapplication.

A numerical scheme is an equation of the following form:

S.h; t; x; uh.t; x/; Œuh�t;x/ D 0 for .t; x/ in Ghnft D 0g (13.3)

uh.0; x/ D u0.x/ in Gh \ ft D 0g (13.4)

where h D .�t;�x/ , Gh D �tf0; 1; : : : ; nT g��xZN is the grid, uh stands for theapproximation of u on the grid, uh.t; x/ is the approximation uh at the point .t; x/,and Œuh�t;x represents the value of uh at other points than .t; x/. Note that uh can beboth interpreted as a function defined at the grid points only or on the whole space.Indeed if one knows the value of uh on the mesh, a continuous version of uh can beconstructed by linear interpolation.

The first and crucial condition in the Barles–Souganidis framework is:

Monotonicity S.h; t; x; r; u/ � S.h; t; x; r; v/ whenever u � v.The monotonicity assumption can be weakened. This was indeed the case in the

previous chapter. We only need it to hold approximately, with a margin of error thatvanishes to 0 as h goes to 0.

Consistency For every smooth function �.t; x/,

limh!0;.n�t;i�x/!.t;x/;c!0

S.h; n�t; i�x;˚.t; x/C c; Œ˚.t; x/C c�t;x/

D ˚t C F.t; x; ˚.t; x/;D˚;D2˚/:

The final condition is:

Stability For every h > 0, the scheme has a solution uh which is uniformly boundedindependently of h.

Theorem 13.2. Assume that the problem (13.1) and (13.2) satisfies the strongcomparison principle for bounded functions. Assume further that the scheme (13.3),(13.4) satisfies the consistency, monotonicity, and stability properties. Then, itssolution uh converges locally uniformly to the unique viscosity solution of (13.1),(13.2).

13.2 First Examples

13.2.1 The Heat Equation: The Classic Explicitand Implicit Schemes

First, let me recall the classic explicit and implicit schemes for the heat equation:

ut � uxx D 0in.0; T � � R: (13.5)


u.0; x/ D u0.x/ (13.6)

and verify that these schemes satisfy the required properties. It is well known thatthe analysis of the linear heat equation does not require the machinery of viscositysolutions. Our intention here is to understand the connection between the theoryfor linear parabolic equations and the theory of viscosity solutions. More precisely,our goal is to verify that the standard finite difference approximations for the heatequation are convergent in the Barles–Souganidis sense.

The Standard Explicit Scheme

unC1i � uni�t

D uniC1 C uni�1 � 2uni�X2

:

Since, this scheme is explicit, it is very easy to compute at each time step nC 1 thevalue of the approximation .unC1

i /i from the value of the approximation at the timestep n, namely .uni /i .

unC1i D uni C�t

�uniC1 C uni�1 � 2uni

�X2

�:

Note that we may define the scheme S by setting:

S.�t;�x; .nC 1/�T; i�x; unC1i ; Œuni�1; uni ; uniC1�/

D unC1i � uni�t

� uniC1 C uni�1 � 2uni�X2

:

Let us now discuss the properties of this scheme. Clearly, it is consistent with theequation since, formally, the truncation error is of order two in space and orderone in time. Let us recall how one can calculate the truncation error for a smoothfunction u with bounded partial derivatives. Simply write the Taylor expansions

uniC1 D uni C ux.n�t; xi /�X C 1

2uxx.n�t; xi /�X

2 C uxxx1

6�X3

C 1

24uxxxx�X

4 C�X4�.�X/

and

uni�1 D uni � ux.n�t; xi /�X C 1

2uxx.n�t; xi /�X

2 � 1

6uxxx�X

3

C 1

24uxxxx�X

4 C�X4�.�X/

13.2 First Examples 205

Then, adding up the two expansions, subtracting 2uni from the left- and right-handsides, and dividing by �X2, one obtains

uniC1 C uni�1 � 2uni�X2

D uxx C 1

12uxxxx�X

2 C o.�X2/;

and thus the truncation error for this approximation of the second spatial derivativeis of order 2. Similarly the expansion

unC1i D uni C ut .n�t; xi /�t C 1

2ut t .n�t; xi /�t

2 C�t2�.�t/

yieldsunC1i � uni�t

D ut .n�t; xi /C 1

2ut t�t C�t�.�t/:

The truncation error for the approximation of the first derivative in time is of order1 only (for more details about computation of truncations errors, see the book byThomas [40]).

Furthermore, the approximation S is monotone if and only if S is decreasing inuni ; u

niC1 and uni�1. First of all, it is unconditionally decreasing with respect to both

uni�1, and uniC1. Secondly, it is only decreasing in uni if the following CFL conditionis satisfied:

�1C 2�t

�X2� 0

or equivalently

�t � 1

2�X2:

The Standard Implicit Scheme

For many financial applications, the explicit scheme turns out to be very inaccuratebecause the CFL condition forces the time step to be so small that the roundingerror dominates the total computational error (computational error = rounding errorC truncation error). Most of the time, an implicit scheme is preferred because it isunconditionally convergent, regardless of the size of the time step. We now evaluatethe second derivative at time .nC 1/�t instead of time n�t ,

unC1i � uni�t

D unC1iC1 C unC1

i�1 � 2unC1i

�X2:

Implementing an algorithm allowing to compute the approximation is less obvioushere. This discrete equation, may be converted into a linear system of equationsand the algorithm will then consist in inverting a tridiagonal matrix. The truncationerrors for smooth functions are the same as for the explicit scheme, and theconsistency follows from this analysis.


We claim that for any choice of the time step, the implicit scheme is monotone.In order to verify that claim, let us rewrite the implicit scheme using the notation S :

S.�t;�x; .nC 1/�T; i�x; unC1i ; ŒunC1

i�1 ; uni ; u

nC1iC1 �/

D unC1i � uni�t

� unC1iC1 C unC1

i�1 � 2unC1i

�X2:

Since S is decreasing in uni ; unC1iC1 , and unC1

i�1 , the implicit scheme is unconditionallymonotone.

13.2.2 The Black–Scholes–Merton PDE

The price of a European call u.t; x/ satisfies the degenerate linear PDE

ut C ru � 12�2x2uxx � rxux D 0 in .0; T � � .0;C1/

u.0; x/ D .x �K/C:

The Black–Scholes–Merton PDE is linear and its elliptic operator is degenerate. Thefirst derivative ux can be easily approximated in a monotone way using a forwardfinite difference

�rxux � �rxi unC1iC1 � unC1

i

�x:

One can, for instance, implement the implicit scheme

S.�t;�x; .nC 1/�T; i�x; unC1i ; ŒunC1

i�1 ; uni ; u

nC1iC1 �/

D unC1i � uni�t

C runC1i � 1

2.i�x/2

unC1iC1 C unC1

i�1 �2unC1i

�X2�ri�x unC1

iC1 � unC1i

�x:

13.3 A Nonlinear Example: The Passport Option


It is an interesting example of a one-dimensional nonlinear HJB equation. I presentonly briefly the underlying model here and refer to the article [41] for more detailsand references. A passport option is an option on the balance of a trading account.The holder can trade an underlying asset S over a finite time horizon Œ0; T �. Atmaturity, the holder keeps any net gain, while the writer bears any loss. The number

13.3 A Nonlinear Example: The Passport Option 207

of shares q of stock held is bounded by a given number C . Without any loss ofgenerality, this number is commonly assumed to be 1 (the problem can be solved infull generality by using the appropriate scaling). The stock S follows a geometricBrownian motion with drift � and volatility � , r is the risk-free interest rate, � isthe dividend rate, rt is the interest rate for the trading account, and rc is its cost ofcarry rate. The option price V.t; S;W /, which depends on S and on the accumulatedwealth W in the trading account, solves the PDE

�Vt C rV � .r ��/SVS � supjqj�1

��..� � r C rc/qS � rtW /VW

C1

2�2S2.VSS C 2qVSW C q2VWW /

�D 0

V.T; S;W / D max.W; 0/:

Next, one can reduce this problem to a one-dimensional equation by introducing thevariable x D W=S and the new function u satisfying V.T � t; S;W / D Su.t; x/.The PDE for u then reads

ut C�u � supjqj�1

f..r �� rc/q �.r �� rt /x/ux C 1

2�2.x�q/2uxxg

u.0; x/ D max.x; 0/:

Note that, in this example, the solution is no longer bounded but grows at mostlinearly at infinity. The Barles and Souganidis [4] framework can be slightlymodified to accommodate the linear growth of the value function at infinity.

When the payoff is convex, it is easy to see that the optimal value for q is eitherC1 or �1. When the payoff is no longer convex, the supremum may be achievedinside the interval at q� D x � .r��rc/ux

�2uxx. For simplicity, we consider only the

convex case.

13.3.2 Finite Difference Approximation

To simplify further, we focus on a simple case: we assume that r � � � rt D 0 andr��rc < 0. This equation is still fairly difficult to solve because the approximationscheme must depend on the control q.

ut C�u � max

�.r �� rc/ux C 1

2�2.x�1/2uxx ;

� .r �� rc/ux C 1

2�2.xC1/2uxx

�

u.0; x/ D max.x; 0/


One can easily construct an explicit monotone scheme by using the appropri-ate forward or backward finite difference for the first partial derivative. Often,this type of scheme is called “upwind” because you move along the directionprescribed by the deterministic dynamics b.x; ˛�/ corresponding to the optimalcontrol ˛� and pick the corresponding neighbor. For instance, for the passportoption, the dynamics is

For q� D 1; b˛�

.t; x/ D q�.r � � � rc/ D .r � � � rc/ < 0

For q� D �1; b˛�

.t; x/ D �.r � � � rc/ > 0;

and the corresponding upwind finite differences are

For q� D 1; ux � D�uni

For q� D �1; ux � DCuni ;

where we used the standard notations

D�uni D uni � uni�1�x

;DCuni D uniC1 � uni�x

:

Then the scheme reads

unC1i �uni�t

C �uni � max

(.r �� rc/uni � uni�1

�xC1

2�2.xi�1/2 uniC1Cuni�1 � 2uni

�x2;

�.r �� rc/uniC1 � uni�x

C 1

2�2.xiC1/2 uniC1 C uni�1 � 2uni

�x2

)D 0:

This scheme clearly satisfies the monotonicity assumption under the CFLcondition

�t � 1

� C jr��rc j�x

C �2 maxfmaxi fi�x�1g2;maxi fi�xC1g2g�x2

:

Approximating the first spatial derivative by the classic centered finite difference,

i.e., ux � uniC1�uni�12�x

, would not yield a monotone scheme here.Note that this condition is very restrictive. First of all, as expected, �t has to

be of order �x2. Furthermore, �t also depends on the size of the grid through theterms .i�x � 1/2; .i�x � 1/2 and even approaches 0 as the size of the domaingoes to infinity. In this situation, we renounce using the above explicit scheme andreplace it by the fully implicit upwind scheme which is unconditionally monotone.

13.4 The Bonnans–Zidani [7] Approximation 209

unC1i � uni�t

C �unC1i � max

(.r �� rc/unC1

i � unC1i�1

�x

C1

2�2.xi�1/2 unC1

iC1 C unC1i�1 � 2unC1

i

�x2;�.r �� rc/unC1

iC1 � unC1i

�x

C1

2�2.xiC1/2 unC1

iC1 C unC1i�1 � 2unC1

i

�x2

)D 0:

Inverting the above scheme is challenging because it depends on the control. Thiscan be done using the classic iterative Howard algorithm which we describe belowin a general setting. However, it may be time-consuming to compute the solution ofa nonlinear finite difference scheme, i.e., invert an implicit scheme using an iterativemethod.

13.3.3 Howard Algorithm

We denote by unh; unC1h the approximations at time n and nC 1. We can rewrite the

scheme that we need to invert as

min˛

fA˛hunC1h � B˛

h unhg D 0:

Step 0: Start with an initial value for the control ˛0. Compute the solution v0h ofA˛0h w � B

˛0h unh D 0.

Step k ! k C 1: Given vkh, find ˛kC1 minimizing A˛hvkh � B˛h unh. Then compute

the solution vkC1h of A

˛kC1

h w � B˛kC1

h unh D 0.

Final step: If jvkC1h � vkhj < �, then set unC1

h D vkC1h .

13.4 The Bonnans–Zidani [7] Approximation

Sometimes, for a given problem, it is very difficult or even impossible to find amonotone scheme. Rewriting the PDE in terms of directional derivatives instead ofpartial derivatives can be extremely useful. For example, in two spatial dimensions,a naive discretization of the partial derivative vxy may fail to be monotone. Infact, approximating second-order operators with crossed derivatives in a monotoneway is not easy. You actually need to be able to interpret your second-order termas a directional derivative (of a linear combination of directional derivatives) andapproximate each directional derivative by the adequate finite difference. In otherwords, you need to “move in the right direction” in order to preserve the ellipticstructure of the operator.


Here is, for instance, a naive approximation of vxy (assume�x D �y):

vxy � viC1;jC1 C vi�1;j�1 � viC1;j�1 � vi�1;jC14�x2

:

It is consistent but clearly not monotone (the terms vi�1;jC1; viC1;j�1 have the wrongsign).

Instead, let us follow Bonnans and Zidani [7]. Consider the second-orderderivative

L˛˚.t; x/ D t r.a˛.t; x/D2˚.t; x//

and assume that the coefficients a˛ admit the decomposition

a˛.t; x/ DXˇ

a˛ˇˇˇT ;

where the coefficients a˛ˇ are positive. The operator can then be expressed in termsof the directional derivativesD2

ˇ D t rŒˇˇTD2�

L˛˚.t; x/ DXˇ

a˛ˇ.t; x/D2ˇ˚.t; x/:

Finally, we can use the consistent and monotone approximation for eachdirectional derivative

D2ˇv.t; x/ � v.t; x C ˇ�x/C v.t; x � ˇ�x/ � 2v.t; x/

jˇj2�x2 :

In practice, if the points x C ˇ�x, x � ˇ�x are not on the grid, you needto estimate the value of v at these points by simple linear interpolation betweentwo grid points. Of course, you have to make sure that the interpolation procedurepreserves the monotonicity of the approximation.

Comments

• In all the above examples, I only consider the immediate neighbors of a givenpoint ..n C 1/�t; i�x/, namely, .n�t; i�x/, .n�t; .i � 1/�x/, .n�t; .i C1/�x/, ..n C 1/�t; .i � 1/�x, and .n C 1/�t; .i C 1/�x/. Sometimes, it isworth considering a larger neighborhood and picking neighbors located furtheraway from ..n C 1/�t; i�x/. It is particularly useful for the discretization of atransport term with a high speed, when information “travels fast.”

• The theoretical accuracy of a monotone finite difference scheme is quite low.The Barles–Jakobsen theory [3] predicts a typical rate of 1=5 (jhj 15 where h Dp�x2 C�t and an optimal rate of 1=2. Sometimes, higher rates are reported in

practice (first order).

13.6 Variational Inequalities and Splitting Methods 211

13.5 Working in a Finite Domain

When one implements a numerical scheme, one cannot work on the whole spaceand must instead work on a finite grid. Consequently, one has to impose some extraboundary conditions at the edges of the grid. This creates an additional source oferror and even sometimes instabilities. Indeed, when the behavior at infinity is notknown, imposing an overestimated boundary condition may cause the computedsolution to blow up. If the behavior of the solution at infinity is known, it is thenrelatively easy to come up with a reasonable boundary condition. Next, one cantry to prove that the extra error introduced is confined within a boundary layer ormore precisely decreases exponentially as a function of the distance to the boundary(see [5] for a result in this direction). Also, one can perform experiments to ensurethat these artificial boundary conditions do not affect the accuracy of the results, byincreasing the size of the domain and checking that the first six significant digits ofthe computed solution are not affected.

13.6 Variational Inequalities and Splitting Methods

13.6.1 The American Option

This is the easiest example of variational inequalities arising in finance, and it givesthe opportunity to introduce splitting methods. We look at the simplified VI: u.t; x/solves

max.ut � uxx; u � .t; x// D 0 in .0; T � � R (13.7)

u.0; x/ D u0.x/: (13.8)

This PDE can be approximated using the following semi-discretized scheme

Step 1. Given un, solve the heat equation

wt � wxx D 0 in .n�t; .nC 1/�t� � R (13.9)

w.n�t; x/ D un.x/; (13.10)

and set

unC 12 .x/ D w..nC 1/�t; x/:

Step 2.

unC1.x/ D inf�

unC 12 .x/; ..nC 1/�t; x/

�:


It is quite simple to prove the convergence of a splitting method using theBarles–Souganidis framework. There are many VI arising in quantitative finance,in particular in presence of singular controls, and splitting methods are extremelyuseful for this type of HJB equations. We refer to the guest lecture by H. M. Sonerfor an introduction to singular control and its applications.

References

1. P. Bank, D. Baum, Hedging and portfolio optimization in financial markets with a large trader.Math. Finance. 14, 1–18 (2004)

2. G. Barles, Solutions de Viscosite des Equations de Hamilton-Jacobi. Mathematiques &Applications (Springer, New York, 1994)

3. G. Barles, E.R. Jakobsen, Error bounds for monotone approximation schemes for parabolicHamilton-Jacobi-Bellman equations. Math. Comput. 76, 1861–1893 (2007)

4. G. Barles, P.E. Souganidis, Convergence of approximation schemes for fully nonlinear secondorder equations. Asymptot. Anal. 4, 271–283 (1991)

5. G. Barles, C. Daher, M. Romano, Convergence of numerical schemes for parabolic equationsarising in finance theory. Math. Model Meth. Appl. Sci. 5, 125–143 (1995)

6. J.-M. Bismut, Conjugate convex functions in optimal stochastic control. J. Math. Anal. Appl.44, 384–404 (1973)

7. F. Bonnans, H. Zidani, Consistency of generalized finite difference schemes for the stochasticHJB equation. Siam J. Numer. Anal. 41, 1008–1021 (2003)

8. U. Cetin, R. Jarrow, P. Protter, Liquidity risk and arbitrage pricing theory. Finance Stochast. 8,311–341 (2004).

9. U. Cetin, M. Soner, N. Touzi, Option hedging under liquidity costs. Finance Stochast. 14(3),317–341 (2010).

10. U. Cetin, R. Jarrow, P. Protter, M. Warachka, Pricing options in an extended black-scholeseconomy with illiquidity: theory and empirical evidence. Rev. Financ. Stud. 19, 493–529(2006).

11. P. Cheridito, M. Soner, N. Touzi, Small time path behavior of double stochastic integrals, andapplication to stochastic control. Ann. Appl. Probab. 15, 2472–2495 (2005)

12. P. Cheridito, M. Soner, N. Touzi, The multi-dimensional super-replication problem undergamma constraints. Annales de l’Institut Henri Poincare, Serie C: Analyse Non-Lineaire. 22,633–666 (2005)

13. P. Cheridito, M. Soner, N. Touzi, N. Victoir, Second order backward stochastic differentialequations and fully non-linear parabolic pdes. Comm. Pure Appl. Math. 60, (2007)

14. M.G. Crandall, H. Ishii, P.-L. Lions, User’s guide to viscosity solutions of second order partialdifferential equations. Bull. Am. Math. Soc. 27, 1–67 (1992)

15. J. Dugundji, Topology (Allyn and Bacon series in Advanced Mathematics, Allyn and Baconedt.) (1966)

16. N. ElKaroui, M. Jeanblanc-Picque, Controle de processus de markov, Seminaire de Proba-bilites Springer Lecture Notes in Math. vol. 1321 (1988), pp. 508–541

17. N. ElKaroui, R. Rouge, Pricing via utility maximization and entropy. Math. Finance. 10,259–276 (2000)

N. Touzi, Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE,Fields Institute Monographs 29, DOI 10.1007/978-1-4614-4286-8,© Springer Science+Business Media New York 2013

213

214 References

18. N. ElKaroui, S. Peng, M.-C. Quenez, Backward stochastic differential equations in fiannce.Math. Finance. 7, 1–71 (1997)

19. A. Fahim, N. Touzi, X. Warin, A probabilistic numerical method for fully nonlinear parabolicPDEs. 21(4), 1322–1364 (2011)

20. W.H. Fleming, H.M. Soner, Controlled Markov Processes and Viscosity Solutions (Springer,New York, 1993)

21. H. Foellmer, P. Leukert, Quantile hedging. Finance Stochas. 3, 251–273 (1999)22. Y. Hu, P. Imkeller, M. Muller, Utility maximization in incomplete markets. Ann. Appl. Probab.

15, 1691–1712 (2005)23. I. Karatzas, S. Shreve, Brownian Motion and Stochastic Calculus, 2nd edn. (Springer, New

York, 1991)24. I. Karatzas, S. Shreve, Methods of Mathematical Finance, 2nd edn. (Springer, New York, 1999)25. N. Kazamaki, Optimal Stopping and Free-Boundary Problems. Lectures in Mathematics

(Springer, Berlin 1579)26. M. Kobylanski, Backward stochastic differential equations and partial differential equations

with quadratic growth. Ann. Appl. Probab. 28, 558–602 (2000)27. N.V. Krylov, Controlled Diffusion Processes (Springer, New York, 1977)28. H. J. Kushner, P.G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous

Time (Springer, New York, 1992)29. R.C. Merton, Lifetime portfolio selection under uncertainty: the continuous-time model. Rev.

Econ. Stat. 51, 247–257 (1969)30. R.C. Merton, Optimum consumption and portfolio rules in a continuous-time model. J. Econ.

Theor. 3, 373–413 (1971)31. M.-A. Morlais, Equations Differentielles Stochastiques Retrogrades a Croissance Quadratique

et Applications. Ph.D. thesis, Universite de Rennes, 200732. E. Pardoux, S. Peng, Adapted solution of a backward stochastic differential equation. Syst.

Contr. Lett. 14, 55–61 (1990)33. D. Pooley, P. Forsyth, K. Vetzal, Numerical convergence properties of option pricing pdes with

uncertain volatility. IMA J. Numer. Anal. 23, 241–267 (2003)34. P.J. Reny, On the existence of pure and mixed strategy nash equilibria in discontinuous games.

Econometrica. 67, 1029–1056 (1999)35. R.T. Rockafellar, Convex Analysis (Princeton University Press, Princeton, 1970)36. H.M. Soner, N. Touzi, Dynamic programming for stochastic target problems and geometric

flows. J. Eur. Math. Soc. 4, 201–236 (2002)37. M. Soner, N. Touzi, The dynamic programming equation for second order stochastic target

problems. SIAM J. Contr. Optim. 48(4), 2344–2365 (2009)38. H.M. Soner, N. Touzi, J. Zhang, Wellposedness of second order backward sdes. Probability

Theory and Related Fields 153, 149–190 (2012)39. R. Tevzadze, Solvability of backward stochastic differential equation with quadratic growth.

Stochast. Process. Appl. 118, 503–515 (2008)40. J. Thomas, Numerical Partial Differential Equations: Finite Difference methods (Texts in

Applied Mathematics) (Springer, New York, 1998)41. J. Wang, P. Forsyth, Maximal use of central differencing for hamilton-jacobi-bellman pdes in

finance. Siam J. Numer. Anal. 46, 1580–1601 (2008)42. T. Zariphopoulou, A solution approach to valuation with unhedgeable risks. Finance Stochast.

5, 61–82 (2001)43. R. Zvan, K.R. Vetzal, P.A. Forsyth PDE methods for pricing barrier options. J. Econom.

Dynam. Control 24, 1563–1590 (2000)

Date post:	01-Dec-2021
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

Fields Institute Monographs

Documents