This article was downloaded by: [193.146.32.73] On: 02 May 2014, At: 21:45Publisher: Institute for Operations Research and the Management Sciences (INFORMS)INFORMS is located in Maryland, USA
Operations Research
Publication details, including instructions for authors and subscription information:http://pubsonline.informs.org
Dynamic Airline Revenue Management with Multiple Semi-Markov DemandShelby Brumelle, Darius Walczak,
To cite this article:Shelby Brumelle, Darius Walczak, (2003) Dynamic Airline Revenue Management with Multiple Semi-Markov Demand.Operations Research 51(1):137-148. http://dx.doi.org/10.1287/opre.51.1.137.12796
Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions
This article may be used only for the purposes of research, teaching, and/or private study. Commercial useor systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisherapproval. For more information, contact [email protected].
The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitnessfor a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, orinclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, orsupport of claims made of that product, publication, or service.
Please scroll down for article—it is on subsequent pages
INFORMS is the largest professional society in the world for professionals in the fields of operations research, managementscience, and analytics.For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org
When a customer requests a discount fare, the airline must decide whether to sell the seat at the requested discount or to hold the seatin hope that a customer will arrive later who will pay more. We model this situation for a single-leg flight with multiple fare classes andcustomers who arrive according to a semi-Markov process (possibly nonhomogeneous). These customers can request multiple seats (batchrequests) and can be overbooked. Under certain conditions, we show that the value function decreases as departure approaches. If eachcustomer only requests a single seat or if the requests can be partially satisfied, then we show that there are optimal booking curves whichdecrease as departure approaches. We also provide counterexamples to show that this structural property of the optimal policy need nothold for more general arrival processes if the requests can be for more than one seat and must be accepted or rejected as a whole.
Received July 1999; revision received April 2001; accepted November 2001.Subject classifications: Dynamic programming/optimal control: semi-Markov. Industries: Transportation/shipping, airline revenuemanagement. Probability: stochastic model application.
Area of review: Transportation.
1. INTRODUCTION
Revenue management is concerned with making efficientuse of a given fixed resource that becomes worthless after agiven time. It uses controls such as booking or sales limitsat various price levels. Revenue management is particularlyimportant in businesses, such as airlines, with low-marginoperations where an improvement of even a fraction of apercentage point can mean a significant increase in profit.In the early literature on the subject, many models are
static, only allow use of inventory information, and assumethat lower-fare classes book first. In that category thepapers by Brumelle et al. (1990) and Brumelle and McGill(1993) offer an analysis; Robinson (1995) removes the lastassumption.One of the first published dynamic models (two customer
types) with potential airline revenue management appli-cations seems to be Gerchak et al. (1985), and, to somedegree, Banerjee and Viswanathan (1989). A more com-prehensive approach was presented later in Lee and Hersh(1993), and then in Subramanian et al. (1999) and Lauten-bacher and Stidham (1999) (all in discrete time with mul-tiple classes, the latter comments on extensions to Poissonarrivals and splittable batches). In a related field, Mamer(1986) has structural results for optimal asset liquidationwith semi-Markov arrival process homogeneous in time.An excellent review of the literature, along with examplesof applications of revenue management in areas other thanthe airline industry, can be found in McGill and van Ryzin(1999).
A number of fairly recent publications investigates var-ious types of dynamic Poisson models in continuoustime, some with nonhomogeneous intensities. In Kleywegtand Papastavrou (1998), Chatwin (1999) (birth and deathmodel), Lewis et al. (1999), Liang (1999), and Zhao (1999),requests are required to be single. This is also the case inthe dynamic pricing formulation of Gallego and van Ryzin(1994), Zhao and Zheng (2000), Zhao (1999), Chatwin(2000), and Feng and Xiao (2000).Multiple accept/reject requests are allowed (but no over-
booking) in Banerjee and Viswanathan (1989), Lee andHersh (1993), Van Slyke and Young (2000) (results onpiecewise constant optimal policies), and Kleywegt andPapastavrou (2001). In each article authors notice that somestructural properties break down. In that regard we presentsimpler examples with only two customer types. In addi-tion, we demonstrate that those structural properties breakdown even if we allow requests to be partially satisfied aswe run out of inventory. For a more general nonhomoge-neous arrival process and with inventory of just one seatwe show that more time to departure need not translate intohigher optimal expected revenue. We also include a coun-terexample to a claim by Lee and Hersh (1993) that a criti-cal time property holds when accept/reject requests are formore than one seat.With the exception of two-period cases in Brumelle et al.
(1990) and in examples of Chatwin (1999), the arrival pro-cess considered in the literature is, exclusively, either adiscrete-time process with request types independently cho-sen at each arrival, or it is a natural extension thereof: Each
fare type follows an independent Poisson process. As suchit is not well suited to situations where demands are depen-dent.The model with time-nonhomogeneous semi-Markov
arrivals presented here includes all of the above processesand, in addition, it does allow for modelling of other arrivalprocesses with demand correlated between classes. It cov-ers, for instance, a discrete-time, multiperiod arrival pro-cess with multiple demand changing in a Markovian wayfrom period to period, and where decisions are made at theend of each period.The last model can be solved by standard backward
induction. In general, however, the optimality equationleads to a system of integro-differential or, in some cases,differential equations. With no-shows and overbooking only(no cancellations), such a system can be efficiently solvedby successive iterations. Since an optimal dynamic policyis sought, this can be done offline (precomputed). To fore-cast a semi-Markov arrival process normally requires moreeffort. But with the continuing shift to Internet-based reser-vation systems the situation can only improve, since thosemake it easier to record more or different types of infor-mation: for example, request time and fare type, regardlessof whether accepted or not.We further extend previous work by allowing a plausi-
ble form of no-shows and overbooking and by consideringseveral types of multiple requests: splittable, accept/reject,modified accept/reject. We identify the splittable batches asthe case where existence of booking curves and, under fur-ther conditions, their monotonicity, is obtained naturally.To demonstrate structural properties we mainly use an
established method of successive approximations, cf. Hin-derer (1985) and Mamer (1986). We believe it is moreintuitive and offers a greater degree of analytical simplicitythan tools from intensity control theory used in most pub-lished papers with Poisson arrivals. We also use pathwisearguments, but in an entirely different way than in Zhaoand Zheng (2000) or Zhao (1999)—in demonstrating time-monotonicity rather than discrete concavity of the valuefunction. Our main objective is to study structural proper-ties of the optimal value function and the optimal dynamicpolicy that facilitate implementation.
1.1. Optimality of Booking-Limit-Type Policy
By means of several examples, Chatwin (1999) questionswhether an optimal policy is always of a booking-limit typewhen demands are dependent between periods and point toa negative answer. The examples are set in discrete timewith two decision periods and with only one (fixed andknown) fare-class booking per period. The size of demandin the second period depends on the size of demand in thefirst period.The examples correspond to the full information setting
where the airline knows demand in the first period and,based on this, determines how many seats to allocate tosatisfy it. Our results then yield that the optimal policy is
of the booking-limit type with the booking limit dependingon the type of the current demand (i.e., the size of demandin the first period in this specific case, since the fare isfixed). For example, given that demand in the first periodis for 20 seats, the optimal booking limit might indeed bedifferent than the one corresponding to first-period demandof 30 seats.Policies of a different type do arise in some situa-
tions. When request type (fare and batch) does not becomeknown when making the decision and requests can be mul-tiple, optimal booking-limit-type policies usually do notobtain. A partial information model best describes thismore general situation. It encompasses both the traditionalairline yield management (with accept/reject decisions) anddynamic pricing models, both of which share a numberof similar structural properties under certain conditions. Inthat respect, and with a plausible form of cancellations,Walczak (2001) has some results and examples.
2. THE AIRLINE SEAT INVENTORYMANAGEMENT PROBLEMAS A FINITE HORIZON SEMI-MARKOVDECISION PROCESS
One major (and common in the literature) assumptionwhich we make, is that the arrival process can be definedindependently of any seat sale policy. That is, the decisionswhether or not to sell seats to specific customers do notaffect the arrival process. For example, customers request-ing, but denied, a discount will not return later. As a con-sequence of this assumption, we can model the arrival pro-cess independently of the decision-theoretic structure, anddo so in the next subsection. In the subsequent subsectionwe introduce the additional structure needed for a (possiblynonhomogeneous) semi-Markov decision process.
Some Notation. To avoid overloading parentheses, theclosed interval from a to b is denoted by
�—–�a�b, as used by
Feller (1966). The minimum of two numbers, say a and b,is sometimes written as a∧ b and the maximum as a∨ b.The positive part of a number a is a+ = a∨ 0. We willalso abuse vector notation by writing ����� = ������ if� = ����. The terms increasing and decreasing will beused in their weak sense in preference to nondecreasingand nonincreasing, respectively.
2.1. The Arrival Process
A customer is characterized by his or her fare type � =����, where � is a fare class and is the number of seatsrequested or batch size. Given a fare type �, we will some-times write �� and � to represent the fare class and batchsize associated with �. Let � denote the set of possiblecustomer fare types. The set of fare classes is denoted by� and the set of possible batch sizes is denoted by �. Notethat � ⊆�×�. It is sometimes convenient to allow a zerobatch size to introduce some event which is not really acustomer arrival. (See the example in §5.1.) For example,
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
Brumelle and Walczak / 139
when booking is initiated at time −T , we generally assumethat there is a customer arrival at this point. Since therewill generally not be a real arrival at time −T , we can usean arrival of type ���0�.We assume that our arrival process is a nonhomogeneous
semi-Markov process ���n�Tn��n= 0�1�2� � � � �, where Tn
is the arrival time and �n is the type of the nth customer.We use basic definitions of the nonhomogeneous semi-Markov processes as laid out in Janssen and De Dominicis(1984), and refer the reader to Cinlar (1975) for com-parison with (homogeneous) semi-Markov and Markov-renewal processes.The semi-Markov property is that there are epochs in the
process (arrival epochs in our case) at which the past andfuture are conditionally independent given the current state.More precisely, we assume that
holds for T0 = −T and for all fare types � ∈ �, n =0�1�2� � � � and t � 0.In accordance with Janssen and De Dominicis (1984),
we impose certain conditions on the probability transitionfunction P��′� s � �� t� �= Pr��n+1 = �′� Tn+1 � s � �n =��Tn = t�. In Janssen and De Dominicis, condition (c)requires that the probability has total mass one for eachfare type � and time t. We have relaxed this condition toallow for discounting if desired. Note that if the transitionfunction P��′� s ��� t� only depends on s− t, then we havethe usual homogeneous semi-Markov process.
Assumption 2.1. (a) Let P�s� t� be the matrix whose����′� component is P��′� s ��� t�. Assume that P�s� t� isa measurable map as a function of s �−T and t �−T ;
(b) If s < t, then P��′� s � �� t� = 0 for each � and �′,so that time moves forward.
(c) For each ��� t�∑�′∈�
lims→�P��′� s � �� t� � 1�
2.2. The Finite Horizon Semi-MarkovDecision Process
Having defined an arrival process, we now show how ourproblem fits into a nonhomogeneous semi-Markov decisionprocess framework. We refer to the model developed in thissection as the semi-Markov decision model.The inventory level or number of seats available, �, takes
values in � = �−�� � � � �−1�0�1� � � � � c�, where c is thecapacity of the plane and negative inventory indicates over-booking. The state of the system when a customer arrivesis the inventory level found by that customer together withthe customer type and the time of arrival. Thus the statespace is � = � ×�×� �−T�0.For each state ����� t�, let � be the action space
and D����� t� ⊂ � the set of admissible actions. � �=������ t� a� ∈� ×� � a ∈D����� t�� is the constraint set.
If a ∈ D����� t� seats are sold to a customer who foundthe system in state ����� t�, then the revenue generated isgiven by the value of the revenue function r����� t�a�. Inthe next subsection, a revenue function will be describedwhich corresponds to a simple model with overbookingand no-shows. We also introduce there some attributes ofthe revenue function which will be needed in later sectionsof the paper. We assume throughout that the action setsare finite and that the constraint set � is measurable withrespect to the Borel �-field of � ×�. This is the case if,for example, the action sets only depend on inventory or thebatch size. The revenue function is assumed to be boundedand measurable. The finiteness assumption can sometimesbe replaced by compactness as in Denardo (1967), and theboundedness assumption can be relaxed using the tech-niques from Mamer (1986) or Schellhaas (1980). However,in our context these assumptions are not restrictive.
Assumption 2.2. The revenue function is bounded andmeasurable. The action space is finite and the constraintset is measurable.
A policy is a rule which determines the action to be takenas a function of the information available, which is the his-tory of the process up to the current time. The rule mightinvolve choosing an action according to some probabilitydistribution. Let � be a set of all measurable policies. Apolicy can be very complex. Fortunately, under certain con-ditions specified in Theorem 2.1 (also see Denardo 1967)one can show that a policy of a simple form will be opti-mal. We call a policy simple if it is deterministic and onlydepends on the current state, not on the history of how itarrived at that state. We assume that there exists at least onesuch policy. Such policies are sometimes called stationary.However, in our context “stationary” might be misleadingbecause time is included as part of the state space.Denardo (1967) uses a Banach space of bounded and
real-valued functions and, wherever necessary, circumventsmeasurability and integrability questions (cf. Example 3,ibid.) We modify his approach slightly by working in the(Banach) subspace of bounded and measurable real-valuedfunctions, and using measurable policies. To this end wehave imposed measurability requirements on the revenuefunction, probability transition function, and the constraintset, and then use a selection theorem to prove that a mea-surable version of optimal policy exists.We now describe the probability transition function Q,
which governs the dynamics of the semi-Markov decisionprocess. Suppose that the process is in state ������ t�and choose admissible action a. Then the probability thatthe next arrival occurs before time s, is of customer type��′�′�, and finds inventory level �′ is
Q��′� �′�′� s � ����� t� a�
�=
P��′�′� s � ��� t� if �′ = �−a�
0 otherwise�
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
140 / Brumelle and Walczak
If a customer buys a seats of type � = ���� at timet when the inventory level is �, then the revenue gen-erated is r����� t�a�, provided that a is an admissibleaction in D����� t� and −T � t � 0. For a given policy�, the stochastic process consisting of the states visitedand the actions chosen is well defined. Let v������ t� bethe expected total revenue generated by policy �, giventhat there is an arrival of a customer of type � at time t(−T � t � 0) who finds inventory level �,
v������ t� �= E
[N t�0!∑n=0
r��n��n�Tn����n��n�Tn��
∣∣∣∣�0 = ���0 = ��T0 = t
]�
For a bounded and measurable revenue function and a mea-surable policy �, that expression is well defined under asuitable contraction assumption and it is a unique solutionto the functional equation v� = r + ∫
v� dQ. The optimalvalue function is v∗����� t� = sup�∈� v������ t�.Under certain conditions which will be established later
in Theorem 2.1, v∗ is the unique solution to the optimalityequation
v����� t� = maxa∈D����� t�
{r����� t�a�
+ ∑�′∈�
∫ 0
tv��−a��′� s�P��′�ds � �� t�
}� (1)
Operator notation will be used in the sequel. Note thatthe value of any policy � is bounded since the revenuefunction is bounded by Assumption 2.2 and the total capac-ity of the plane is finite. In considering value functions itis indeed enough to consider functions in the space � ofbounded, Borel-measurable, real-valued functions definedon the state space � . Define the operators and whicheach map � into � by
� v������t�= maxa∈D�����t�
�r�����t�a�+v��−a���t�� (2)
and
� v������ t� = ∑�′∈�
∫ 0
−tv����′� s�P��′�ds � �� t�
for t ∈ � �−T�0
for each state ����� t� ∈ � . Using operator notation, theoptimality equation becomes v = � �v.Apart from the decomposition of the dynamic program-
ming operator into operators and , the developmentand tools used here are standard and follow Denardo (1967)to a great degree. Therefore, we will omit most proofs andrefer the interested reader to Brumelle and Walczak (1999)for details.Let �·� denote the sup norm on �. For an opera-
tor, say � on �, define the modulus of � to be ��� =
supf � g∈� ��f −�g�/�f −g� (cf. Denardo 1967). An oper-ator with modulus less than 1 is a contraction. The func-tion 1�·� is the function in � which has the value 1 for allstates.An additional assumption is needed to ensure that the
optimality equation has a unique solution. We will assumethat there is positive probability, uniform for all states, thatthere will be no more than N arrivals between the timeof the state and the departure at time 0. Note that if weare in state ����� t�, then � 1������ t� is the probabilitythat an arrival will occur before the plane departs. If theseprobabilities are uniformly strictly less than 1, then is acontraction. Hence we can state our assumption as an N -step contraction assumption.
Assumption 2.3. For some integer N , N is a contractionoperator.
This assumption, which will be in force throughout therest of this paper, is not at all restrictive in practice. Forexample, suppose that the interarrival distributions are suchthat there is at least probability & that an interarrival willexceed ' regardless of the state. Then N is a contrac-tion for N � T/'. Our results are similar to those ofSchellhaas (1980) and Mamer (1986), who prove analo-gous results in the time-homogeneous semi-Markov setting.When Assumptions 2.1, 2.2, and 2.3 are satisfied, the fol-lowing theorem obtains.
Theorem 2.1. The optimal value function, v∗, is theunique solution of the optimality equation and v∗ =limn→�� �n0. An optimal policy, �∗, can be obtainedby setting ������ t� equal to an action which maximizesr����� t�a�+ v∗��−a��� t�.
Because of Assumption 2.2 the operator maps �into � and we can restrict ourselves to measurable policiesand measurable value functions. The constraint set can beshown to be measurable when, as in our case, the set ofadmissible actions depends only on the countable compo-nent of the state variable.
Lemma 2.1. The optimal value function v∗ is measurable.Moreover, there exists a measurable version of the optimalpolicy �∗.
2.3. Attributes of the Revenue Functionand Action Sets with Examples
The revenue generated by action a in state ����� t�is described by the function r����� t�a�. The incre-mental revenue from one additional seat in inventory is(r����� t�a�= r����� t�a�−r��−1��� t�a�. The incre-mental value of selling a seats versus not selling any seatsis 'r����� t�a� = r����� t�a�− r����� t�0�. Note thatwe do not assume that r����� t�0�= 0. Some attributes ofthe revenue function, which are plausible and will be usedin the following sections, are now defined.
Increasing in inventory: r����� t�a� is increasing ininventory � for each ��� t� and action a.
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
Brumelle and Walczak / 141
Discretely concave: (r����� t�a� is decreasing ininventory � for each ��� t� and action a.
Decreasing in time: The revenue function r����� t�a�is decreasing in time t for each ����� and action a.
Affine form: r��+a��� t�a� = r����� t�0�+a���� t�for each state ����� t� and admissible action a, where���� t� does not depend on �.
Increasing increments: The revenue function is affine,���� t� is decreasing in time t, and 'r����� t�1� isincreasing in t. Note that 'r����� t�a� increasing in t isthe property used to name this attribute.
Ordered fare classes: The fare classes can be ordered sothat �′ > � implies that r����′�� t� a� � r������ t�a�.
2.3.1. Accept/Reject and Splittable Batches. We con-sider two possible cases—splittable batches and accept/reject. In the accept/reject case D������ t� = �0�� if � �+� and is �0� otherwise; i.e., the whole batch mustbe accepted, or nothing. The overbooking pad � is thelimit on the number of seats which can be overbooked, andhas been introduced in Subramanian et al. (1999). It alsoensures that the revenue function is uniformly bounded.In the case of splittable batches, D������ t� =
�0�1�2� � � � �∧ ��+���; i.e., we decide how many seatsout of the batch of size to accept. Note that D does notdepend on the time of arrival or the fare class, but only on����.
2.3.2. Models with Terminal Reward. A simple modelwhich accommodates no-shows and overbooking is adoptedfrom McGill (1989) and Subramanian et al. (1999). In thismodel, as in the previous one, we denote both the amountof the fare and the fare class of fare type � by �� to sim-plify the notation.A booked customer will show up at departure with prob-
ability p. This probability does not depend on the cus-tomer’s fare type or time of arrival, and the customersdecide whether or not to show independently. So if the finalinventory at departure is �, then the number of customerswho show up at departure, X���, has a binomial distribu-tion with parameters c−� and p.A customer who does not show up is entitled to a refund
�,. With no discounting, and thanks to the assumed inde-pendence, we can replace fare �� offered by a customerof type � by the expected fare �� = �� − �1−p��,. It isapparent that the refund , to no-shows can depend on faretype and booking time.The airline is, in addition, penalized for the customers
who cannot be seated in the plane. It is reasonable thatthis penalty is a convex-increasing function, say f �·�,of the number of customers who were denied boarding, X���−�c+��!+. Let L0���=−E f � X���−�c+��!+�!represent the expected loss due to overbooking. Using thearguments in McGill (1989) and Subramanian et al. (1999),one can show that L0 is concave and increasing.The simple model with overbooking and, more generally,
models with terminal reward do not immediately fit into thesemi-Markov decision framework presented in §2.2. One
standard way to deal with that is to include the effect of theterminal reward L0��� in the expected immediate revenueby adding a term L0��−a�Pr(current arrival is last beforedeparture � �� t) to the revenue function r����� t�a� andthus obtain an equivalent model that generates the sameexpected total revenue under the same policy (cf. Schell-haas 1980). Precisely for that reason we had to allow arevenue function more general than simply a��.The equivalent semi-Markov formulation for problems
with terminal reward may result in the revenue functionthat is nonzero when a customer’s offer is rejected. Espe-cially when r����� t�0� is negative, certain monotonicityproperties in time might be awkward to demonstrate. Inthis case we can use a shift transformation (Stidham 1994).The transformed problem generates total expected revenuethat differs from the original one under the same policyby a function that can be computed separately. In our casewe use the shift transformation that removes fixed costsdue to the terminal reward, so that only variable costs areincluded in the value function. The transformation involvesL, the expected revenue given state ����� t�, under the pol-icy rejecting the current and all subsequent arrivals untildeparture. We define it formally as follows.
����� t�. � �n is the n-fold composition of the operator with itself with the convention that � �0f ����� t� =f ����� t�.
The transformed problem has revenue function of theform r����� t�a�− r��−a��� t�0�− �L����� t�−L��−a��� t��, which is zero when we reject a customer (a= 0).Then, as argued in Stidham (1994), vtr
�—the value of apolicy � in that problem is related to v� , the value of �in the original formulation by v� = vtr
� +L. It follows alsothat (v� = (vtr
� +(L.Thus, if it happens that L is decreasing in time, and we
can show that the optimal expected revenue v∗tr is decreas-
ing, then so will v∗, the original optimal value function.Similarly, with (L and (v∗
tr both decreasing we obtain thatso is (v∗. In the simple model with overbooking, the valueof a policy rejecting all arrivals is constant in time, so allof the time-monotonicity results from §§3.1 and 4.2 apply.
3. BASIC RESULTS
Adopting notation from Sun (1992), define the functionv0 = v∗, where v∗ is the optimal value function. Notethat v0������ t� is the expected value of having inventorylevel � just after the demand from a customer at time t wasdealt with, given that there was an arrival of type ����at time t and that an optimal policy is used from then on.Lemma 3.1 follows immediately from the optimality equa-tion, which can be written as v∗ = v0.
Lemma 3.1. Suppose there is a request from a customer oftype ���� who arrives at time t to find an inventory levelof �.
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
142 / Brumelle and Walczak
(a) If the request is splittable, then it is optimal to sella∗ seats, where
a∗ ∈ argmaxa∈D������ t�
�r����� t�a�+v0��−a���� t���
(The largest action in the set will be chosen by convention.)(b) If the request is accept/reject, then it is optimal to
accept the request for seats if
v0������ t�−v0��−���� t� � 'r����� t���
An example of an arrival process, which satisfies thehypothesis in the following corollary is when customersarrive according to a (single) renewal process and the faretype of each arrival is chosen independently with constantprobabilities over the whole period
� �−T�0� The corollaryfollows from the previous lemma by noting that v0 willnot depend on the fare class � of the current arrival andfrom the hypothesis that the fare classes order the revenuefunction.
Corollary 3.1. Suppose that the probability transitionfunction for the arrival process does not depend on thefare class of the current state and that the fare classesare ordered by the revenue function. In this case, the fareclasses under the optimal policy are nested. That is, if weaccept seats from the fare type ����, then we accept atleast the same number of seats from all fare types ��′��with higher fares for the same inventory level, batch sizeand time (i.e., �′ > �).
We use Lemma 3.1 to give a version of a theorem due toSun (1992) in the semi-Markov setting. Similar results areshown in Lautenbacher and Stidham (1999) and Gallegoand van Ryzin (1994).
Theorem 3.1. Suppose that the revenue function isincreasing in inventory �. Then v∗����� t� is increasing in� for each ��� t�.
Proof. Let �∗ be an optimal policy and for inventory lev-els � > �′ let k = �−�′. Define policy �0 as the pol-icy, which ignores k seats in the inventory but otherwisebehaves like the optimal policy �∗. That is,
�0����� t� ={
�∗��−k��� t� for � � k�
0 otherwise�
First, suppose that the revenue function is constant in �.Then for each realization, policies �∗ and �0 will choosethe same actions and generate the same revenue. With theassumption that the revenue function is increasing in �,policy �0 might generate more revenue than �∗ because ofthe higher inventory levels. The optimal policy will gener-ate expected revenue at least as large as �0. Hence
v∗����� t� � v�0����� t� � v∗��′��� t�� � (3)
Theorem 3.1 states that when using the optimal pol-icy, having more seats is better than having fewer. This
result is true much more generally than for the nonhomo-geneous arrival process which we are using, since the argu-ment which leads to Equation (3) holds for each realization.Thus, the theorem holds for essentially any arrival processwhich is not affected by the policy being used and withimmediate revenue increasing in inventory.
3.1. Monotonicity of the Optimal ValueFunction with Respect to Time
A question similar, but more difficult, to that answered inTheorem 3.1 is whether v∗����� t� is decreasing in t. Itseems as though the affirmative should be correct, since ina longer interval one should be able to make more money.There are some cases where applying a policy “as if lesstime were available” in the spirit of Lemma 3.1 works. Twoexamples, which are familiar in the stochastic processes lit-erature, are the nonhomogeneous Poisson process and thehomogeneous semi-Markov processes. The last case wegeneralize to a process with a stochastically increasing tran-sition function.Define a nonhomogeneous Poisson arrival process as an
arrival process with a probability transition density of theform
P��′�ds � �� t� = /�′�s� exp 0�s�−0�t�! ds� (4)
where /� is the intensity of fare type � and 0�t� =∫ 0t
∑�′∈� /′
��s�ds.Note that the probability transition function does not
depend on the current fare type �, that this process hasindependent increments, and that probability of two ormore arrivals at any time is zero.This type of nonhomogeneous Poisson arrival process
has been used in Gerchak et al. (1985) and in many recentpapers. Lee and Hersh (1993) used a discrete version of thismodel with the exponential interarrival distribution replacedby a geometric.
Theorem 3.2. Assume that the arrival process is eithernonhomogeneous Poisson or is homogeneous semi-Markov.In the homogeneous case assume that the revenue functionr����� t�a� is decreasing in t. In both cases assume thatr����� t�0� � 0 and that 0 ∈ D����� t�. Then v∗����� t�is decreasing in t.
Proof. Suppose that t′ < t � 0 and let h = t− t′.In the homogeneous semi-Markov case, the arrival pro-
cess starting in state (���� t′) will have the same distribu-tion as the arrival process starting in state (���� t) exceptfor the shift in the time parameter by h.So if we start the process in state (���� t′), run it for
�t� units of time using the optimal policy �∗ as though westarted at time t, then the arrival process will have the samedistribution as the process started in state (���� t) and thecorresponding revenues will be higher because the revenuefunction is decreasing in time. Thus, after taking expecta-tions, the value received will be no less then if starting instate (���� t).
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
Brumelle and Walczak / 143
More formally, let �0����� s� = �∗����� s + h� fort′ � s � t′ + �t� and �0����� s� = 0 otherwise. Becausethe arrival process is homogeneous, the distribution of thearrival process ���n�Tn��n = 0�1�2� � � � � given the initialstate ����� t′� is identical to the distribution of the arrivalprocess given the initial state ����� t�. Because r����� s+h�a�� r����� s�a� the corresponding immediate revenuesare higher when starting at t′. For s > t′ + �t�, we reject allarrivals, but do not lose revenue as r����� t�0� � 0. Con-sequently, v∗����� t′� � v�0����� t′� � v∗����� t�.If the arrival process is nonhomogeneous Poisson, then
the following argument shows that we can couple the end(from t on) of the process which starts at t′ with the processstarting at t. This is in contrast to the homogeneous case,where the first parts of trajectories were matched.By hypothesis, the probability of more than two arrivals
at any given time is zero, so let N�t′� s� count the arrivalsduring the interval
�t′� s, given that the process starts in state
(���� t′). It also holds that the distribution of the excess attime t of the process starting at time t′, i.e., TN�t′� t�+1− t,is the same as the distribution of T1 − T0 of the processstarting at time t. In fact, the realizations of the process���n�Tn − t�� n = N�t′� t�+1�N �t′� t�+2� � � � � starting instate ����� t′� can be coupled with those of the process���n�Tn��n= 1�2� � � � �� which starts in the state ����� t�.Construct a policy �0, which sells no seats between t′
and t, as follows:
�0����� s� =
�∗����� t� for s = t′�
0 for t′ < s � t�
�∗����� s� for t < s�
Now consider a realization of the process starting instate ����� t′�, which has been coupled with that of aprocess starting in ����� t�. For that realization, sincer����� t�0� � 0, the policy �0 achieves at least the samerevenue starting in state ����� t′� as does policy �∗ start-ing in state ����� t�. Hence v�0����� t′�= v∗����� t�. Theoptimal policy might do better than �0, so v∗����� t′� �v∗����� t�. �
The homogeneous semi-Markov process can be gener-alized somewhat. Define the probability transition functionP to be stochastically increasing if
∫ x
−tP��′�ds � �� t� is
decreasing in t for each time x, and fare types � and�′. This stochastic ordering was introduced by Veinott(1974) in an inventory context (also see Brumelle and Vick-son 1975) and is known as first-degree stochastic order-ing. A homogeneous semi-Markov process is stochasticallyincreasing.
Theorem 3.3. Assume that the arrival process is stochas-tically increasing and that 0 ∈ D����� t� for each state����� t�. Also assume that for each ����� and action a,r����� t�a� is decreasing in t and that r����� t�0� � 0.Then v∗����� t� is decreasing in t.
Proof. Suppose that v ∈� is nonnegative and decreasingin t. Because the probability transition function is stochas-tically increasing, v is also decreasing in t (the proof usesthe standard argument from Brumelle and Vickson 1975).Using the hypothesis on the revenue function, it is easy toshow that v is decreasing in t. Hence, v is decreasingin t. It follows from Theorem 2.1 that v∗ = limn→�� �n0is decreasing in t. �
4. BOOKING CURVESFOR SPLITTABLE BATCHES
Some policies are more convenient to work with than oth-ers. A class of policies of interest are those which canbe described by a set of integer-valued booking curves,�∗��� t�, one curve for each fare type. Given a set of book-ing curves, the corresponding policy in a state ������ t�sells no seats if � � �∗���� t�; and otherwise (for split-table batches) sells the minimum of the batch size and�−�∗���� t�. The booking curves thus determine a criti-cal inventory level for each fare type and time, below whereno sales take place and above where as many seats as pos-sible are sold. A policy of this class will be referred to asa booking-curve policy.A booking curve that is decreasing in t for fixed � is
particularly convenient computationally because it is piece-wise constant and can thus be characterized by criticaltimes, t∗����� =max�t � �∗��� t� = ��, at which the criti-cal inventory level changes. Only a finite number of criticaltimes are needed—one for each inventory level up to thecapacity of the plane and each fare type.One might also consider the class of policies which can
be described by a set of critical times, t∗�����. Given aset of critical times and a state ����� t�, the correspondingpolicy is as follows: Satisfy the customer’s request if t >t∗�����. As pointed out by Lee and Hersh (1993), a setof critical times might exist for an optimal policy whichcannot be determined by a set of booking curves if t∗�����is not decreasing in � for each fare type �.In the remainder of this section, batches are assumed
to be splittable. We first show a concavity property ofthe optimal value function, which ensures that an opti-mal set of booking curves will exist. Then we considertwo special arrival processes, nonhomogeneous Poisson andstochastically increasing, for which there is an optimal setof booking curves which are decreasing as the departureapproaches.
4.1. Existence of Booking Curvesfor Splittable Batches
We begin this section by deriving some structural propertiesof the optimal value function and its auxiliary function v0 = v, assuming that batches are splittable.
Definition 4.1. A function v ∈� is discretely concave ifv����� t�− v��−1��� t� is decreasing in � for each faretype � and time t.
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
144 / Brumelle and Walczak
Results similar to the following lemma can be traced toStidham (1978), Langen (1982), Lautenbacher and Stidham(1999), and Lewis et al. (1999).
Lemma 4.1. Assume that the revenue function is discretelyconcave in inventory and has affine form. Then v����� t�is discretely concave whenever v ∈� is.
Proof. Let �= ���� and suppose v����� t� is discretelyconcave in �, so that (v����� t� = v����� t� − v�� −1��� t� is decreasing in �. Let �∗ = min�� � ���� t� �(v����� t�+(r����� t�0��− 1 if the set over which theminimum is defined is not empty, and �∗ = c otherwise.If the current state is ����� t� and a seats are sold, then
the expected value is
r����� t�a�+v��−a��� t� = a���� t�+ r��−a��� t�0�
+v��−a��� t�� (5)
Because v�·��� t� and r�·��� t�0� are discretely concave,the definition of �∗ implies that ���� t� � (v����� t�+(r����� t�0� for � > �∗ and that the reverse inequal-ity holds for � � �∗. It follows that for � > �∗, thechange in expected revenue from selling one seat, ���� t�−(v����� t�−(r����� t�0�, is not negative. So one shouldsell ��−�∗�+ ∧ seats. Similarly, no seats should be soldif � � �∗. Hence,
Using the affine form of the revenue function, the increment
( v����� t�
=
(v����� t�+(r����� t�0� if � � �∗����� t� if �∗ < � � �∗ +�
(v��−��� t�+(r��−��� t�0�
if � > �∗ +�
(7)
can be seen to be decreasing and v is discretelyconcave. �
Lemma 4.2. The operator preserves discrete concavity.
Proof. This is clear as is linear on � and transformsnonnegative functions into nonnegative functions. �
The two previous lemmas immediately provide the fol-lowing result.
Lemma 4.3. If the revenue function is discretely concavewith affine form, then the composed operators and preserve discrete concavity.
Theorem 4.1. If the revenue function is discretely concavewith affine form, then the optimal value function v∗����� t�and the corresponding auxiliary function v0= v∗ are eachdiscretely concave. Moreover, there is an optimal booking-curve policy.
Proof. The function which is 0 for all states is discretelyconcave, so � �n0 is also discretely concave by thepreceding lemma. By Theorem 2.1, v∗ = limn→�� �n0.Hence, v∗ is discretely concave. Since preserves concav-ity, v0 = v∗ is also discretely concave. �
4.2. Decreasing Booking Curves
The construction in Lemma 4.1 of �∗ leads to a set ofbooking curves, which under certain conditions establishedin this section, will be decreasing.Let �∗
v��� t� be the inventory level computed in theproof of Lemma 4.1, where v is some discretely concavefunction in �. Then ��∗
v��� ·��� ∈ �� is a set of book-ing curves which correspond to the policy �v������ t� =��−�∗
v���� t��+ ∧.Define a function v ∈ � to have decreasing increments
if (v����� t� = v����� t�−v��−1��� t� is decreasing int for each �����.
Lemma 4.4. Suppose that the revenue function has increas-ing increments and that (r����� t�0�� 0 is decreasing intime, in addition to the hypotheses in Lemma 4.1. Then vhas decreasing increments, ( v � 0, and �∗
v is a decreas-ing set of booking curves whenever v ∈ � is increasingand discretely concave, and has nonnegative and decreas-ing increments.
Proof. The definition of �∗ in Lemma 4.1 can be rewrittenas �∗
v��� t� = min�� � 'r����� t�1� � (v����� t�� − 1.Since 'r����� t�1� is increasing and (v����� t� isdecreasing in t (by increasing increments and the assump-tion of the theorem, respectively), it follows that �∗
v��� t�is decreasing in t. In addition, each of the three expres-sions on the right-hand side of Equation (7) is nonnegativeand decreasing. Hence (� v������ t� is nonnegative anddecreasing in t. �
4.2.1. Decreasing Booking Curves for StochasticallyIncreasing Processes
Theorem 4.2. Suppose that the revenue function satisfiesthe hypotheses of Lemma 4.4 and that the arrival pro-cess has a stochastically increasing probability transitionfunction. Let v0 = v∗. Then the increments (v∗����� t�and (v0����� t� are decreasing in t, nonnegative, and thebooking curves are decreasing.
Proof. Let v����� t� be the function which is 0 forall states. This function has decreasing (constant) incre-ments in t because of the assumption, and is (clearly)greater or equal to 0. Since maps decreasing non-negative functions into decreasing nonnegative functions
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
Brumelle and Walczak / 145
as argued in Theorem 3.3, v has decreasing incre-ments. By the previous lemma, � �v also has decreas-ing increments and �( �v � 0. Consequently, v∗ =limn→�� �n0 has decreasing increments. Also, v0 hasdecreasing increments. �
4.2.2. Decreasing Booking Curves for NonhomogeneousPoisson Processes
Theorem 4.3. Suppose that the revenue function hasincreasing increments and that r����� t�0� � 0, in addi-tion to the hypotheses in Lemma 4.1. If the arrival pro-cess is nonhomogeneous Poisson, then the increments(v0����� t� are decreasing in t and the booking curvesare monotone.
Proof. As in Theorem 3.2, suppose that t′ < t � 0. Let �0
be an inventory level, which is at least −� + 1 but other-wise arbitrary, and let �0 be an arbitrary fare type. It isenough to show that (v0��0��0� t
′� � (v0��0��0� t�.For t′ � s < 0 define V���� s� to be a random variable
representing the revenue generated during the interval—–�s�0
using the policy � given that there is inventory � at times and given that there was an arrival of type �0 at time t′.More explicitly, let 4 = ���0� t0�� ��1� t1�� � � � � be a real-ization of the arrival process with t0 = t′. Let N�t′� s� be thecounting variable introduced in the proof of Theorem 3.2.Then the value of V���� s� for the particular realization 4is
V���� s�4� =n=N�t′�0�∑
n=N�t′� s�+1r��n��n� tn����n��n� tn���
where �n+1 = �n −���n��n� tn� for n � N�t′� s�+ 1 and�N�t′� s�+1 = �. In the proof of Theorem 3.2, we argued thatthe distribution of the arrival process from time t onward,i.e., the distribution of the arrival process ���n�Tn− t�� n=N�t′� t�+1�N �t′� t�+2� � � � � starting in state ��0� t
′� is thesame as that of the process ���n�Tn��n= 1�2� � � � �, whichstarts in the state ��0� t�. In fact, the realizations of oneprocess can be coupled with each other. So consider a real-ization 4 of the process starting in state ����� t′�, whichhas been coupled with that of a process starting in ����� t�.Then one way of describing (v∗��0��0� t� is that it is
the expected value of the first ticket which would be soldto a customer if the inventory at t were �0 but would notbe sold if the inventory at t were �0− 1. With this char-acterization in mind, define Y to be the random variablerepresenting the time at which the extra seat is sold. So forthe realization 4, Y takes the value
Y �4�=min�tn � tn � t��∗��n��n� tn� > �∗��n−1��n� tn��
for 4 = ���0� t′�� ��1� t1�� � � � � and where the inventory at
time t is �0. If the set defining Y is empty, we set Y = 1 orany other positive value, since no rewards are accumulatedafter time 0 (the departure). Note that because of discreteconcavity, the booking curves exist and
(V�∗��0� t� = rY � (8)
where rY represents the additional revenue from the extraseat sold at time Y .Define the policy � for the arrival process starting at t′
with inventory �0 as follows: Set one seat aside and applythe optimal policy as though the initial inventory were �0−1. However, once the time Y is reached, sell the seat whichwas set aside. Note that for each realization, the optimalpolicy might sell the extra seat before Y , but never afterbecause of discrete concavity. More explicitly, the policy� is
������ s� =
�∗��−1��� s� for s < Y �
�∗��−1��� s�+1 for s = Y �
�∗����� s� for s > Y �
The policy � is not Markov because after time t it needsto keep track of what the inventory would be, assumingthat there was inventory �0 at time t in order to tell if Yhas been reached, as well as to keep track of the actualstates. Nevertheless, � is in the class of policies � whichare admissible.Note that from the construction of �, and by the fact
that r����� t�0� � 0
V���0� t′�−V�∗��0−1� t′� � rY � (9)
which by Equation (8) is equal to (V�∗��0� t�. Takingexpectations, we have
E V���0� t′� � ��0��0� t
′�!−v0��0−1� t′��(v0��0��0� t��
which is less than or equal to (v0��0��0� t′� since the pol-
icy � is not necessarily optimal. �
5. EXAMPLES
In this section we present three examples, which show thatthe optimal policy may lose the desired critical time prop-erty because (v0��� t� is not necessarily decreasing in t.The revenue function is r������ t�a� = a� and � = 0 sothat there is no overbooking. The first is a counterexam-ple for Theorem 3 in Lee and Hersh (1993) (accept/rejectcase).The second example shows that even in the case of cus-
tomers who request single seats, the critical time propertymight not hold. A variation of this example has indepen-dent increments.The third example shows that even if we relax the
accept/reject constraint slightly, the function (v0��� t�is not necessarily decreasing, and moreover, the criticaltime property need not hold. In this modification of theaccept/reject constraint we continue to require that batchesbe either accepted or rejected as a unit whenever there isenough inventory to satisfy the request. However, a requestcan be partially satisfied if there is not enough inventory tosatisfy it totally. So only the last batch which receives anyseats can be split.
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
146 / Brumelle and Walczak
5.1. A Discrete-Time Counterexamplefor Lee and Hersh
Lee and Hersh (1993) state that in the discrete-time case,when the arrivals occur in batches that are either acceptedor rejected as a whole, the optimal policy can be formulatedin terms of critical times for each class and batch size. Theirproof claims that the expected increment (v0������ t�=v0������ t�−v0��−���� t� is decreasing in time t forfixed-fare type ���� and fixed inventory level �. Whilefor the single arrival case their work (ibid. pp. 255–258) isconsistent with our previous results, there seems to exist acounterexample both in discrete time and continuous timefor the accept/reject batch case.Consider a three-period decision model with two fare
types in which batches are either accepted or rejected. Faretype 1 customers have fare �1 = 1 and batch size two (�1 =��1�2��. Fare type 2 customers have fare �2 = 0�5 and batchsize one (�2 = ��2�1)). We also will use the fictitious faretype 0 (�0 = �0�0�) as mentioned in §2.1 to simplify themodelling of independent periods.Let −T =−2 so that the periods will correspond to times
−2, −1, and 0. Suppose that for −2� t < −1 and all faretypes �
P��′�−1 � �� t� ={
p2 = 0�5 if �′ = �2�
1−p2 = 0�5 if �′ = �0�
for −1� t < 0 and all fare types �
P��′�0 � �� t� ={
p1 = 0�3 if �′ = �1�
1−p2 = 0�7 if �′ = �0�
and that all other transitions have probability 0.At time 0 it is optimal to fill any request which can be
filled, so
v�����0� =
2 if � � 2 and � = �1�
0�5 if � � 1 and � = �2�
0 otherwise�
(10)
At time −1 with � = 1 it is optimal to sell a seat toa request from fare type �2 since that is the only way asingle seat can be utilized. If the inventory level is 2, thena request from fare type �2 should be rejected since therewill be no �2 arrival in the future to utilize the single seat,and the immediate revenue of 0�5 is less than the expectedrevenue of 2× 0�3 = 0�6 which is obtained from the twoseats in period 0. The value of two seats at time −1 is thus
v�2���−1� ={2 if � = �1�
0�6 if � = �2�
At time −2 with inventory level 2, a type 2 request forone seat should be satisfied since it yields a revenue of 0.5plus the expected revenue of 0�5× 0�5 for a total of 0.75,which is greater than the expected revenue of 0.6 which is
obtained by rejecting the request. The value of two seats attime −2 is thus
v�2���−2� ={2 if � = �1�
0�75 if � = �2�
There is no critical time for inventory level 2 andfare type �2 since a type 2 request is accepted attime −2, rejected at time −1, and accepted at time0. Also v����1�0� is not discretely concave because(v�1��1�0� = 0 and (v�2��1�0� = 2.We remark that in the homogeneous, discrete-time set-
ting with � chosen independently at each arrival, Papas-tavrou et al. (1996) offer conditions on the joint distributionof � and ensuring that some structural properties are pre-served.
5.2. Counterexample withModified Accept/Reject
We can show that even if we are allowed to split abatch as we run out of inventory, the monotonicity isdestroyed. With this modification, the set of admissibleactions becomes D������ t�= �0�� if ��, and �0���otherwise. In this example each fare type arrives accordingto an independent nonhomogeneous Poisson process, inde-pendently of the others, as described in §3.1. The inten-sity function for fare type � = ���� will be denoted by/�. There are two fare classes with revenues �1 = 1 and�2 = 0�5, respectively. Class 1 customers always request asingle seat. Class 2 customers request either one or twoseats. So there are three fare types with intensity func-tions /11�t� = 1�4 if t �−1 and 0 otherwise, /21�t� = 0 ift �−1�5 and 10 otherwise, and /22�t� = 0 if t �−0�5 and1.2 otherwise.From §2, it follows that the optimal value function v and
the auxiliary function v0 satisfy the optimality equationsfor the nonhomogeneous semi-Markov case—namely, v∗ = v0 and v0 = � �v0. Substituting Equation (4) into theoptimality equations and proceeding in the usual fashion,we obtain a system of backward Kolmogorov ODEs for(v0. Its solutions were generated using the NDSolve routinein Mathematica.
for � � 3. As before the fare type component, �, of thestate has not been indicated since v0 for a nonhomogeneousPoisson arrival process does not depend on it.The plot in Figure 1 shows that (v0�4� t� crosses the �2
level twice, and so the optimal policy does not have thecritical time property.
5.3. Single Requests Without CriticalTime Property
This example shows that the critical time property need nothold even when each arrival has only one request.Customers arrive according to a renewal process with
independent and identically distributed interarrival times,each of which takes the value 2 with probability 0�5, andotherwise takes the value 5.There are two fare types—�1 = �3�1� and �2 = �1�1�.
If an arrival happens at time t, then the type of request ischosen independently of the arrival epochs and of the faretypes of the other arrivals. An arrival at time t requestsfirst class with probability p1�t� and the discount class withprobability p2�t� (p1�t�+p2�t� = 1).The probability that a customer has fare type �1 is
p1�t� = 1 if t � −1 and 0 otherwise. The probability thata customer is fare type 2 is p2�t� = 1−p1�t�, so that lowfares arrive before high.Suppose that the initial inventory at time −T =−4 is 1.
If there is a type 2 customer who arrives at a time −2< t �0, then there will be no more subsequent customers beforedeparture at time 0, so the request should be satisfied.If there is a type 2 customer who arrives at a time −3�
t � −2 and the inventory level is 1, then there will onlybe a type 1 customer with probability 0�5 with an expectedrevenue of �1× 0�5 = 1�5, so the type 2 customer shouldnot be satisfied because its revenue is only �2 = 1.If a type 2 customer arrives at time t <−3 and the inven-
tory level is still 1, then in order for there to be a type 1arrival before departure at time 0, the next two interarrivalswould have to be length 2, which has a probability of 0.25.So the request by the type 2 arrival should be satisfiedbecause its revenue of 1 is larger than the expected revenuefrom a type 1 customer of 0�25�1 = 0�75.
Consequently, the optimal policy does not have the criti-cal time property for an inventory level of 2 since a type 2request is accepted for −3� t �−2 and rejected otherwise.Similar examples can be constructed using the above
idea of a mixture of distributions (one short and one long)for the interarrival time distribution and with low faresbooking before high. An interesting one is to mix a constant0 with an exponential. This gives an arrival process withindependent increments which is not Poisson. By choosingthe parameters suitably, the critical time property will nothold.
6. SUMMARY
This paper considered a continuous-time, finite hori-zon, nonhomogeneous semi-Markov decision model whererequests arrive in batches and overbooking is allowed. Weshow that under certain conditions, the optimal value func-tion has decreasing optimal booking curves which allow thecharacterization of the optimal policy in terms of criticaltimes, one for each pair (inventory, fare of current request).Finally, we provide examples where the optimal value func-tion is not decreasing in time (and no critical times exist)and also a counterexample to Lee and Hersh (1993), wherethe critical times need not exist when the batches have tobe rejected or accepted as a whole. Detailed proofs and afew minor results that were not included here can be foundin the working paper by Brumelle and Walczak (1999).
ACKNOWLEDGMENT
Regrettably Professor Shelby L. Brumelle passed awayshortly before the final version of this paper was completed.
REFERENCES
Banerjee, P. K., B. Viswanathan. 1989. On optimal rationing poli-cies. Canadian J. Admin. Sci. 1–6.
Brumelle, S. L., J. I. McGill. 1993. Airline seat allocation withmultiple nested fare classes. Oper. Res. 41(1) 127–137., R. G. Vickson. 1975. A unified approach to stochastic dom-inance. W. T. Ziemba, R. G. Vickson, eds. Stochastic Modelsin Finance. Academic Press, New York., D. Walczak. 1999a. Dynamic allocation of airline seatinventory with batch arrivals. Working paper, UBC Com-merce, University of British Columbia, Vancouver, BritishColumbia, Canada., D. Walczak. 1999b. Dynamic allocation of airline seatinventory with batch arrivals. T. H. Oum, B. D. Bowen, eds.Proc. 1997 Air Transport Research Group (ATRG) of theWCTR Society, Vol. 3., J. I. McGill, T. H. Oum, K. Sawaki, M. W. Trethe-way. 1990. Allocation of airline seats between stochasticallydependent demands. Transportation Sci. 24(3) 183–192.
Chatwin, R. E. 1999. Continuous-time airline overbooking withtime- dependent fares and refunds. Transportation Sci. 33(2)192–206.. 2000. Optimal dynamic pricing of perishable products withstochastic demand and a finite set of prices. Eur. J. Oper.Res. 125 149–174.
Dow
nloa
ded
from
info
rms.
org
by [
193.
146.
32.7
3] o
n 02
May
201
4, a
t 21:
45 .
For
pers
onal
use
onl
y, a
ll ri
ghts
res
erve
d.
148 / Brumelle and Walczak
Cinlar, E. 1975. Markov renewal theory: A survey. ManagementSci. 21(7) 727–753.
Denardo, E. V. 1967. Contraction mappings in the theory under-lying dynamic programming. SIAM Rev. 9(2) 165–177.
Feller, W. 1966. An Introduction to Probability Theory and ItsApplications, Vol. 2. Wiley, New York.
Feng, Y., B. Xiao. 2000. A continuous-time yield managementmodel with multiple prices and reversible price changes.Management Sci. 46(5) 644–657.
Gallego, G., G. van Ryzin. 1994. Optimal dynamic pricing policyof inventories with stochastic demand over finite horizon.Management Sci. 40(8) 999–1020.
Gerchak, Y., Y. M. Parlar, T. K. M. Yee. 1985. Optimal rationingpolicies and production quantities for products with severaldemand classes. Canadian J. Admin. Sci. 2(1) 161–176.
Hinderer, K. F. 1985. On the structure of solutions of stochas-tic dynamic programs. Proc. Seventh Conf. on ProbabilityTheory (Brasov, 1982). VNU Scientific Press, Utrecht, TheNetherlands, 173–182.
Janssen, J., R. De Dominicis. 1984. Finite nonhomogeneous semi-Markov processes: Theoretical and computational aspects.Insurance Math. Econom. 3(1) 157–165.
Kleywegt, A. J., J. D. Papastavrou. 1998. The dynamic andstochastic knapsack problem. Oper. Res. 46(1) 17–35., . 2001. The dynamic and stochastic knapsack problemwith random sized items. Oper. Res. 49(1) 26–41.
Langen, H.-J. 1982. Applying the method of phases in the opti-mization of queueing system. Adv. Appl. Probab. 14 122–142.
Lautenbacher, C., S. Stidham Jr. 1999. The underlying Markovdecision process in the single-leg airline yield-managementproblem. Transportation Sci. 34 136–146.
Lee, Tak C., M. Hersh. 1993. A model for dynamic airline seatinventory control with multiple seat bookings. TransportationSci. 27(3) 252–265.
Lewis, M. E., H. Ayhan, R. D. Foley. 1999. Optimal admis-sion policies for a multi-class nonstationary queuing system.Working paper, Georgia Institute of Technology, Atlanta, GA.
Liang, Y. 1999. Solution to the continuous time dynamic yieldmanagement model. Transportation Sci. 33(1) 117–123.
Mamer, J. W. 1986. Successive approximations for finite horizon,semi-Markov decision processes with application to asset liq-uidation. Oper. Res. 34(4) 638–644.
McGill, J. I. 1989. Optimization and estimation problemsin airline yield management. Ph.D. thesis, Faculty of
Commerce and Business Administration, University ofBritish Columbia, Vancouver, BC, Canada., G. J. van Ryzin. 1999. Revenue management: Researchoverview and prospects. Transportation Sci. 33(2) 233–256.
Papastavrou, J. D., S. Rajagopalan, A. J. Kleywegt. 1996. Thedynamic and stochastic knapsack problem with deadlines.Management Sci. 42 1706–1718.
Robinson, L. W. 1995. Optimal and approximate control poli-cies for airline booking with sequential nonmonotonic fareclasses. Oper. Res. 43(2) 252–263.
Schellhaas, H. 1980. Markov renewal decision process with finitehorizon. OR-Spektrum 2 33–40.
Stidham, S., Jr. 1978. Socially and individually optimal con-trol of arrivals to a GI/M/1 queue. Management Sci. 241598–1610.. 1994. Successive approximations for Markovian decisionprocesses with unbounded rewards: A review. F. P. Kelly, ed.Probability, Statistics and Optimisation. Wiley, New York.
Subramanian, J., S. Stidham Jr., C. J. Lautenbacher. 1999. Airlineyield management with overbooking, cancellations, and no-shows. Transportation Sci. 33 147–167.
Sun, Xiao. 1992. Airline yield management: A dynamic seat allo-cation model. M.Sc. thesis, Faculty of Commerce, Universityof British Columbia, Vancouver, BC, Canada.
Van Slyke, R., Y. Young. 2000. Finite horizon stochastic knap-sacks with applications to yield management. Oper. Res.48(1) 155–172.
Veinott, A. F., Jr. 1974. Class notes, course on inventory theory.Operations Research Department, Stanford University, Stan-ford, CA.
Walczak, D. 2001. Dynamic modelling approaches to airline reve-nue management. Research monograph (based on author’sPh.D. thesis), Centre for Transportation Studies, Universityof British Columbia, Vancouver, BC, Canada.
Weatherford, L. R., S. E. Bodily. 1992. A taxonomy and researchoverview of perishable-asset revenue management: Yieldmanagement, overbooking and pricing. Oper. Res. 40(5)831–844.
Zhao, W. 1999. Dynamic and static yield management systems.Ph.D. thesis, The Wharton School, University of Pennsylva-nia, Philadelphia, PA., Y. Zheng. 2000. Optimal dynamic pricing for perishableassets with non-homogeneous demand. Management Sci. 46375–388.
![Semi Markov Model for Market Microstructure [P. Fodra, H. Pham] 2013](https://static.documents.pub/doc/80x56/577cc99f1a28aba711a43b3b/semi-markov-model-for-market-microstructure-p-fodra-h-pham-2013.jpg)