1
Deadline Constrained Packet Scheduling in
Wireless Networking
Aditya Dua, Nicholas Bambos
Department of Electrical Engineering
Stanford University
{dua,bambos}@stanford.edu
Abstract
Third generation (3G) of wireless communication networks aim to provide quality-of-service (QoS)
sensitive services to downlink users. Amongst these are real-time multimedia services like video-
conferencing and live media streaming, where the packets tobe delivered to the user have deadlines
associated with them. Downlink packet scheduling plays a key role in efficiently allocating base station
resources to meet the desired level of QoS for various users.In this paper, we study the design of a
downlink wireless packet scheduler that is capable of supporting applications with packet deadlines. We
use a dynamic programming (DP) approach to study a parsimonious model of the scheduling problem.
We propose a heuristic scheduling policy based on the structural properties of the optimal solution to the
DP. We contrast the performance of the proposed scheduler tothat of standard benchmark schedulers via
extensive link-level simulations. The proposed scheduleroffers a significantly better trade-off between
user fairness and system performance.
Index Terms
3GPP, HSDPA, QoS, Packet scheduling, Dynamic Programming.
July 8, 2005 DRAFT
2
I. INTRODUCTION
Third generation (3G) wireless communication systems aim to support quality-of-service
(QoS) based services like interactive multimedia and high-speed data [1]. However, bandwidth
constraints (due to physical and regulatory reasons), power constraints (due to battery limitations)
and the erratic nature of the wireless channel (due to shadowing, fading and mobility) make this
a challenging task for wireless system designers [2].
While traditional digital communication system design approaches (source and channel code
design, constellation design etc.) [3] have been investigated thoroughly, cross-layer design method-
ology has attracted much interest in recent years [4]. This approach tries to exploit the synergy
existing between different layers of the communication protocol stack to achieve more efficient
designs, instead of treating each layer as an individual entity. QoS and channel aware packet
scheduling is an important illustration of the cross-layerdesign approach, which exploits the
interactions between the physical, network and application layers.
Packet scheduling is an important component of the high-speed downlink packet access
(HSDPA) technology, which has been introduced in the 3GPP Release 5 specifications to provide
high data rates and QoS sensitive services to downlink users[5]. In fact, the packet scheduler
has been moved from the radio network controller (RNC) to thebase-station (BS) to support
fast physical layer re-transmissions.
Packet scheduling for wireline networks has been studied extensively over the years. However,
scheduling algorithms designed for wireline networks are not directly applicable to the wireless
scenario, because wireless channels suffer from bursty errors and time and location dependent
capacity. Much work has been done on adapting wireline scheduling algorithms to wireless
networks. See [6] and references therein for a review.
Since the performance of any wireless packet scheduler is affected by random channel fluctu-
ations, it is essential to incorporate channel state information into the scheduling decision. For
instance, such information is available in a 3G cellular system via the channel quality indicator
July 8, 2005 DRAFT
3
(CQI) signal, which is periodically transmitted by the downlink users to the BS. Also, using
other protocol and application related information (backlog states of various queues, link-layer
deadlines on packets, minimum data rate requirements, etc.) can enhance the performance of
the scheduler from a system perspective. From a user perspective, a scheduler must possess an
element offairness, so that each user can get its due share of service. Fair scheduling of packets
without deadlines in wireless networks has been studied in [7]. Also, a scheduler should be able
to provide differentiated QoS to users. For instance, the packets of a user who pays more to
receive service should have higher priority for scheduling. A “good” scheduler should be capable
of achieving a trade-off between system performance and user fairness in a flexible manner and
should have tunable parameters to provide differentiated service to different classes of users and
data.
Wireless schedulers that have received a lot of attention incorporate one or more of the
attributes discussed above. Theround-robinscheduler, while being fair, is insensitive to wireless
channel fluctuations and incapable of providing differentiated QoS to users. Themaximum SNR
scheduler utilizes the available bandwidth efficiently by transmitting to the user with the best
quality channel. However, it is unfair to users with poor channels and can lead to large fluctuations
in QoS experienced by downlink users. The proportional-fair scheduler [8] achieves a trade-off
between system throughput and long-term user fairness. None of these schedulers are suited to
real-time applications with packet deadlines.
The QoS requirements for real-time applications are very different from those of non-real-time
applications. For instance, video-conferencing, which isa real-time variable-bit-rate (rt-VBR)
application has a delay bound of 40-90 ms and an acceptable loss rate of10−3. In contrast, non-
real-time applications like web browsing and file transfer have a stringent acceptable loss rate
requirement of10−8, but can withstand large delays. These numbers clearly indicate the need for
designing packet scheduling algorithms tailored to the requirements of real-time applications.
The earliest-deadline-first(EDF) scheduling policy, also known as theearliest-due-dateor
shortest time-to-extinction(STE) has been shown to be optimal for deadline constrained schedul-
July 8, 2005 DRAFT
4
ing over wireline (error-free) channels under various modeling assumptions. A variant of EDD
applicable to wireless networks, namelyfeasible earliest deadline due(FEDD) is studied in
[9] and shown to perform well for two-state Markovian (ON-OFF) wireless channels. The
EDF scheduling policy and its variants are easily implementable, but inflexible and incapable
of providing differentiated QoS to users. In addition, EDF does not account for variations in
channel conditions and is expected to be unfair to users withpoor channel conditions. In [10],
the authors study downlink scheduling of a mixture of real-time and non-real-time traffic and
show that theexponential rule(EXP-rule) performs well for both types of traffic. In [11], the
authors study deadline constrained scheduling for a revenue based model and show that a simple
greedy algorithm achieves a competitive ratio of12
under certain assumptions on the revenue
function. In [12], the authors consider delay sensitive scheduling based on minimizing a heuristic
cost function that accounts for both channel and queue states.
Our goal in this paper is to design a flexible and easily implementable downlink packet
scheduling algorithm for a scenario in which packets in eachqueue have associated deadlines,
as is the case in multimedia and mobile computing applications. Such constraints can also be
artificially imposed to achieve objectives like load balancing across streams in mobile computing
and networking applications.
We define the scheduling problem and an associated parsimonious model in Section II. In
Section III, we capture the scheduling trade-offs within a dynamic program (DP) framework
[13], [14] and state some key results regarding the structural properties of its optimal solution.
A DP based approach lends itself naturally to the problem of packet scheduling. For instance,
downlink scheduling for non-real time data using a DP formulation is studied in [15]. We study
the asymptotic behavior of the optimal solution in Section IV and construct an approximation to
the optimal solution in Section V. Based on the latter, we propose a heuristic scheduling policy
and evaluate its performance via extensive link level simulations in Section VII. We demonstrate
that under several different scenarios, the proposed scheduler offers a significantly better trade-
off between user fairness and system performance compared to other schedulers. We conclude
July 8, 2005 DRAFT
5
in Section VIII.
II. M ODEL CONSTRUCTION ANDREDUCTION
We consider a time-division multiplexed (TDM) wireless cellular communication system. Time
is slotted into fixed size transmission time intervals (TTI)or slots. There is a queue corresponding
to each downlink user at the base station (BS) which is servedaccording to a first-come first-
served (FCFS) discipline. In each slot, the BS schedules thehead-of-line (HOL) packet of a
non-empty queue for transmission, and selects a modulationand coding scheme (MCS) and a
transmission power for the scheduled packet. The channel for each user exhibits temporal and
spatial random fluctuations. Packets in each queue have associated deadlines, that is, if they
are not (successfully) transmitted before the expiration of the deadline they get dropped. The
scheduled user is notified of the scheduling decision of the BS prior to transmission. We assume
that the notification and transmission occur in the same slot. An instantaneous and error free
feedback indicating a successful/failed transmission is available on the uplink to the BS. Also,
we assume that the scheduler has perfect and most recent knowledge of the queue backlog state
and channel conditions for each user.
Despite the simplifying assumptions, the scheduling problem posed in the aforementioned
form is hard to solve. We can cast it into a DP framework and solve it numerically. However,
such a solution will neither provide any structural insightinto the scheduling problem, nor will
its complexity be desirable from an implementation perspective. In what follows, we construct
a parsimonious reduction of the original problem and derivea heuristic scheduling policy based
on the optimal solution to the reduced problem.
A. Model Reduction
If we were to solve the scheduling problem as posed above using a DP approach, the state
of the system would comprise the deadlines associated with each packet in each queue, in
addition to all the channel states. This would clearly lead to a huge state-space that grows
July 8, 2005 DRAFT
6
exponentially with the number of users and render the problem intractable. However, for delay
sensitive applications like multimedia streaming, packets in a queue exhibit strong temporal
correlation in that they are typically ordered in increasing order of their deadlines. It is important
for packets to meet differential deadlines relative to the packets ahead of them in the queue, rather
than meeting absolute deadlines. For instance, in a lossless multimedia transmission scenario (no
packet dropping), if the current video frame gets delayed, then the deadlines on subsequent frames
in the sequence are also incremented appropriately to accommodate this delay. We refer to these
differential deadlines asinter-packet deadlines(IPD). We leverage the temporal correlation to
achieve a tremendous reduction in the size of the state spaceby using the IPD of the HOL
packet (relative to the preceding HOL packet) as the state ofeach queue.
B. Reduced Problem
With the above reduction in state-space, let us consider a discrete-time system where two
queues,Q1 andQ2 are competing for a single server (transmitter)S to transmit their packets
to receiversR1 and R2 respectively. We assume that each queue hasexactly one packet. In
each time slot,S selects eitherQ1 or Q2 according to some scheduling policy and transmits its
HOL packet. The HOL packet ofQi has an associated deadline, that is, if it is not transmitted
successfully before the deadline expires, it is dropped. Inthe context of the original problem,
this deadline reflects the the inter-packet deadline or IPD (relative to the preceding HOL packet)
associated with the HOL packet ofQi1. We model the channels from the transmitter to each
receiver as independent and identically distributed (i.i.d) random processes, independent of each
other. A simple truncated ARQ mechanism is employed for eachqueue, that is, a packet is
transmitted till it is received successfully or its associated deadline expires, and the success
probability of a re-transmission is unaffected by previoustransmissions. Once the HOL packet
of a queue is successfully transmitted it stays empty and is not a potential candidate for scheduling
1When there is only packet in the queue, the inter-packet deadline (IPD) is same as the (absolute) deadline. Hence, in the
context of the reduced problem we use deadline and IPD interchangeably.
July 8, 2005 DRAFT
7
in subsequent time-slots. Each packet incurs a buffering cost for every time slot it spends in the
queue, and a dropping cost if it is not transmitted successfully prior to the expiry of its deadline.
We re-emphasize that adopting thislocal perspectiveon delay constrained scheduling based
on inter-packet deadlines yields a tremendous reduction inthe size of the state-space.
We now state some definitions that will be used extensively inthe rest of the paper.
Definition: Let Di be the deadline associated with the HOL packet ofQi.
Definition: Let si be the probability of successful transmission of the HOL packet of Qi,
if it is scheduled.
Definition: Let ci be the buffering cost per slot incurred by the HOL packet ofQi.
Definition: Let λi be the dropping cost incurred by the HOL packet ofQi, if it is dropped
due to expiration of its deadline before successful transmission.
Definition: Let ni be the time-to-expiration (TTE) of the original deadlineDi on the HOL
packet ofQi.
Note that by definition,0 ≤ ni ≤ Di.
Definition: Let thestate of the system be the two-tuple(n1, n2).
Let P denote the set of all non-idling packet scheduling policiesthat schedule the HOL packet
of exactly one non-empty queue in each time slot. With the above defined state-space and cost
structure, our goal is to design the optimal packet scheduling policyP⋆ ∈ P that minimizes the
total expected buffering and dropping costs.
The states(n1, 0) ∀n1 ∈ N and (0, n2) ∀n2 ∈ N respectively correspond toQ2 andQ1 being
empty 2. Thus, there is no scheduling decision to be made in these states andQ1 andQ2 are
respectively scheduled. The system dynamics under a candidate scheduling policyP ∈ P, if it
schedulesQ1 in state(n1, n2) ∀n1, n2 ∈ N are described below:
• n1 > 1 andn2 > 1: The transmission is successful with probabilitys1 and the new state
is (0, n2 − 1). The transmission fails with probability(1 − s1), and the new state is(n1 −
2Throughout this paper, we denoteZ0+ = {0, 1, 2, . . .}, N = {1, 2, 3, . . .} andN\{1} = {2, 3, 4, . . .}.
July 8, 2005 DRAFT
8
1, n2 − 1).
• n1 = 1 andn2 > 1: The new state is always(0, n2 − 1). However, the transmitted packet
gets dropped with probability(1 − s1), and a dropping cost ofλ1 is incurred.
• n1 > 1 andn2 = 1: The transmission is successful with probabilitys1 and the new state is
(0, 0), and the transmission fails with probability(1−s1), and the new state is(n1−1, 0). The
HOL packet ofQ2 is dropped in both cases, and a dropping cost ofλ2 is incurred. Note that
the scheduling algorithm terminates in the former case, that is, (0, 0) is an absorbing/stopping
state.
The dynamics of the system state can be similarly explained if P schedulesQ2 in state(n1, n2)
∀n1, n2 ∈ N.
Remarks:Assigning possibly different dropping costs to packets allows the scheduler to
provide differentiated QoS to users and also to packets within the same queue. Delay-sensitive
scheduling policies like EDD are incapable of such differentiation. We illustrate this via simula-
tion results in Section VII. Incorporating buffering costsin the formulation provides additional
degrees of control. In some applications there may be an incentive not merely to meet the
packet deadlines, but transmit as many packets as possible.An example is video transmission
with packets coded using incremental redundancy (IR), where the video quality can be enhanced
if incremental information is transmitted in addition to the base frame before the deadline expires.
Incorporating a buffering cost into the model helps providesuch differentiation between queues,
and also between packets within the same queue. Also, as discussed in Section IV, the buffering
costs determine the behavior of the scheduler when packets are far off from their respective
deadlines.
III. DYNAMIC PROGRAMMING FORMULATION
Definition: Let V (n1, n2) be theexpected cost-to-goin state(n1, n2) if the optimal control
is used, that is, packets are scheduled using the policyP⋆. Thus,V (n1, n2) is the expected cost
(buffering and dropping) of successfully transmitting or dropping the HOL packets ofQ1 and
July 8, 2005 DRAFT
9
Q2 when their times-to-expiration aren1 andn2 respectively.
If n1, n2 ∈ N\{1}, the system transitions from state(n1, n2) to state(0, n2−1) with probability
s1 and to state(n1−1, n2−1) with probability(1−s1) if Q1 is scheduled. The system transitions
to state(n1 − 1, 0) with probability s2 and state(n1 − 1, n2 − 1) with probability (1− s2) if Q2
is scheduled. Thus,V (n1, n2) is given by the recursive equation
V (n1, n2) = min{s1V (0, n2 − 1) + (1 − s1)V (n1 − 1, n2 − 1),
s2V (n1 − 1, 0) + (1 − s2)V (n1 − 1, n2 − 1)} + c1 + c2, n1, n2 ∈ N\{1}. (1)
To simplify notation, we define
α(n1, n2) = V (0, n2 − 1) − V (n1 − 1, n2 − 1), n1, n2 ∈ N\{1}
β(n1, n2) = V (n1 − 1, 0) − V (n1 − 1, n2 − 1), n1, n2 ∈ N\{1}, (2)
and expressV (n1, n2) as
V (n1, n2) = min{s1α(n1, n2), s2β(n1, n2)}+V (n1−1, n2−1)+c1+c2, n1, n2 ∈ N\{1}. (3)
If n1 = 1, n2 ∈ N\{1}, the system always transitions to state(0, n2 − 1) if Q1 is scheduled.
If Q2 is scheduled, the system transitions to state(0, 0) with probability s2 and state(0, n2 − 1)
with probability(1−s2). In the former case the HOL packet ofQ1 gets dropped with probability
(1 − s1), while in the latter case it always gets dropped. The state transitions can be similarly
described ifn1 ∈ N\{1}, n2 = 1. We have,
V (1, n2) = min{V (0, n2 − 1) + (1 − s1)λ1, (1 − s2)V (0, n2 − 1) + λ1} + c1 + c2, n2 ∈ N\{1}
V (n1, 1) = min{(1 − s1)V (n1 − 1, 0) + λ2, V (n1 − 1, 0) + (1 − s2)λ2} + c1 + c2, n1 ∈ N\{1}.
(4)
To fully characterize the cost-to-go function we specify the boundary conditions. The state(0, 0)
July 8, 2005 DRAFT
10
is an absorbing state for the system and has zero cost associated with it.
V (0, 0) = 0
V (0, 1) = c2 + (1 − s2)λ2
V (1, 0) = c1 + (1 − s1)λ1
V (1, 1) = min{(1 − s1)λ1 + λ2, λ1 + (1 − s2)λ2} + c1 + c2. (5)
The behavior of the optimal scheduling policyP⋆ in states(n1, n2) ∀n1, n2 ∈ N\{0} is
captured by the functionγ(n1, n2) : N\{1} × N\{1} → R, which we now define:
Definition: Let γ(n1, n2) = s1α(n1, n2) − s2β(n1, n2), n1, n2 ∈ N\{1}.
Using (3) and the definition ofγ(n1, n2), P⋆ schedulesQ1 in state(n1, n2) ∀n1, n2 ∈ N\{1}
if γ(n1, n2) ≤ 0, andQ2 if γ(n1, n2) > 0.
We now state some results regarding the structural properties of the optimal scheduling policy
P⋆.
Lemma 1:
(a) V (n1, 0) =c1s1
+ (1 − s1)n1
(λ1 −
c1s1
), n1 ∈ N, s1 ∈ (0, 1]
(b) V (0, n2) =c2s2
+ (1 − s2)n2
(λ2 −
c2s2
), n2 ∈ N, s2 ∈ (0, 1].
Proof: See Appendix.
Remarks:V (n1, 0) represents the expected buffering and dropping cost incurred in the
transmission of the HOL packet ofQ1 when it hasn1 > 0 time slots remaining before the
expiry of its deadline. Since our focus is on deadline constrained packet scheduling we expect
the dropping cost to dominate the buffering cost, especially close to deadline expiration. Thus, we
would expect the total expected cost to increase as we get closer to the deadline (n1 decreases)
since the packet dropping probability increases (since thechannel is time invariant and all
transmissions are independent of past transmissions). Now, the sequenceV (n1, 0), n1 ∈ N
July 8, 2005 DRAFT
11
is a decreasing sequence ifc1 < s1λ1 and an increasing sequence ifc1 > s1λ1. Based on the
foregoing discussion, we assume that the problem parameters satisfyc1 < s1λ1. Similarly, we
also assumec2 < s2λ2.
Lemma 2: In the states(n1, 1), n1 ∈ N one of the following is possible
(a) P⋆ schedulesQ1 for all n1 ∈ N.
(b) P⋆ schedulesQ2 for all n1 ∈ N.
(c) There exists an⋆1 > 1, such thatP⋆ schedulesQ1 for 1 ≤ n1 < n⋆
1 andQ2 for n1 ≥ n⋆1.
Proof: See Appendix.
Lemma 3: In the state(1, n2) , n2 ∈ N one of the following is possible
(a) P⋆ schedulesQ1 for all n1 ∈ N.
(b) P⋆ schedulesQ2 for all n1 ∈ N.
(c) There exists an⋆2 > 1, such thatP⋆ schedulesQ2 for 1 ≤ n2 < n⋆
2 andQ1 for n2 ≥ n⋆2.
Proof: The proof is symmetric to the proof of Lemma 2.
Now, we state a key lemma that will enable us to deduce the formof the optimal scheduling
policy P⋆.
Lemma 4:γ(n1, n2 + 1) ≤ γ(n1, n2) ≤ γ(n1 + 1, n2), n1, n2 ∈ N\{1}.
Proof: See Appendix.
Equipped with Lemmas 1, 2, 3 and 4, we are ready to state a result regarding the form of
the optimal scheduling policy for the two-queue single-server deadline constrained scheduling
problem with time-invariant channels. Before that, we havea definition.
Definition: A scheduling policyP ∈ P is called aswitch-over policyif there is a non-
decreasing switch-over functionφ : Z0+ → Z0+ such that it schedulesQ1 in state(n1, n2),
n1, n2 ∈ Z0+ if n2 ≥ φ(n1) andQ2 else [14].
Theorem 1:[Optimality of Switch-over Policy] The optimal packet scheduling policyP⋆ ∈ P
is a switch-over policy for all states(n1, n2), n1, n2 ∈ N.
Proof: See Appendix.
July 8, 2005 DRAFT
12
IV. A SYMPTOTIC BEHAVIOR OF P⋆
From Lemma 4 we have thatγ(n1, n2) is monotonically increasing inn1 for fixed n2, and
monotonically decreasing inn2 for fixed n1. Thus, limn1→∞
γ(n1, n2) and limn2→∞
γ(n1, n2) exist,
even though they may be+∞ and−∞ respectively. From Lemma 1 we have
limn1→∞
V (n1, 0) =c1s1
limn2→∞
V (0, n2) =c2s2. (6)
Lemma 5:
limn1→∞
limn2→∞
γ(n1, n2) = limn2→∞
limn1→∞
γ(n1, n2) = γ⋆.
Proof: See Appendix.
Now, from the definition ofγ(n1, n2) and (6),
limn1→∞
limn2→∞
γ(n1, n2) =s1c2s2
−s2c1s1
+ (s2 − s1) limn1→∞
limn2→∞
V (n1 − 1, n2 − 1)
limn2→∞
limn1→∞
γ(n1, n2) =s1c2s2
−s2c1s1
+ (s2 − s1) limn2→∞
limn1→∞
V (n1 − 1, n2 − 1). (7)
From (7) and Lemma 5 we conclude,
limn1→∞
limn2→∞
V (n1, n2) = limn2→∞
limn1→∞
V (n1, n2) , V ⋆. (8)
As a direct consequence,α(n1, n2) andβ(n1, n2) also have unique successive limitsα⋆ andβ⋆
respectively. In particular,
α⋆ =c2s2
− V ⋆
β⋆ =c1s1
− V ⋆. (9)
Having established the existence of a unique successive limit for the cost-to-go function, we
express the limit as a function of problem parameters, namely, ci, si andλi (i = 1, 2). From the
July 8, 2005 DRAFT
13
definition of V (n1, n2), we note thatV ⋆ must satisfy
V ⋆ = min(α⋆, β⋆) + V ⋆ + c1 + c2
= min
(s1c2s2
− s1V⋆, s2
c1s1
− s2V⋆
)+ V ⋆ + c1 + c2. (10)
We have two cases:
(a) s1c2s2
−s1V⋆ ≤ s2
c1s1
−s2V⋆: In this case, it is optimal to scheduleQ1 for largen1, n2. We
use (10) to getV ⋆ =c1s1
+c2s2
+c1s2
. On substituting this value ofV ⋆ into the inequality,
we obtain the conditionc2s1
≤c1s2
, or equivalently,s1c1 ≥ s2c2.
(b) s1c2s2
− s1V⋆ > s2
c1s1
− s2V⋆: In this case, it is optimal to scheduleQ2 for large n1, n2.
We use (10) to getV ⋆ =c1s1
+c2s2
+c2s1
. On substituting this value ofV ⋆ in the inequality
we obtain the conditionc2s1>c1s2
, or equivalently,s2c2 > s1c1.
Thus, it is optimal to scheduleQi (i = 1, 2) for largen1, n2, if sici = max{s1c1, s2c2}. From
(a) and (b) above, the successive limit of the cost-to-go function is given by
V ⋆ =c1s1
+c2s2
+ min
{c1s2,c2s1
}. (11)
From (7), we obtain
γ⋆ =
c2s2
s1− c1 s1c1 ≥ s2c2
c2 − c1s1
s2s1c1 < s2c2.
(12)
Now, we state another key result regarding the structural properties of the optimal policyP⋆.
Theorem 2:[Saturation of Switch-over Curve]
(a) If s1c1 ≥ s2c2, there existsn⋆1, n
⋆2 ∈ N such thatφ(n1) = n⋆
2 ∀n1 ≥ n⋆1.
(b) If s1c1 < s2c2, there existsn⋆1 ∈ N such thatφ(n1) = ∞ ∀n1 > n⋆
1,
whereφ is the switch-over curve, as defined in Theorem 1.
Proof: See Appendix.
V. APPROXIMATING THE SWITCH-OVER CURVE
We have demonstrated that the optimal packet scheduling policy P⋆ is a switch-over policy.
It is hard to analytically compute the switch-over functionφ. The solution can be obtained via
July 8, 2005 DRAFT
14
uneqlam.eps
Fig. 1. (Approximate) shape of switch-over curves for different scenarios -Clockwise:(a) s1c1 ≥ s2c2, s1λ1 < s2λ2, (b)
s1c1 ≥ s2c2, s1λ1 ≥ s2λ2, (c) s1c1 < s2c2, s1λ1 ≥ s2λ2, (d) s1c1 < s2c2, s1λ1 < s2λ2; • and◦ denote the states in which
it is optimal to scheduleQ1 andQ2, respectively. O and K respectively denote the location of the offset and the knee
numerical methods, however, such an approach is computationally expensive and not scaleable
from an implementation perspective. Thus, it is natural to seek an approximation to the switch-
over curve that can be characterized parsimoniously and computed easily.
Possible forms of the switch-over curve are depicted in Figure 1. The figure suggests that the
switch-over curve can be closely approximated by a piece-wise linear curve with two components,
one with slope0 or π/2 and the other with slopeπ/4. As shown in the figure, the piece-wise
linear approximation can be characterized by two points: the kneeand theoffset. These points
can be computed analytically in terms of the problem parameters, namelysi, ci, λi (i = 1, 2).
In this Section, we derive analytical expressions for thekneeand theoffsetunder the assump-
tion (s1λ1 − c1) > 0 and (s2λ2 − c2) > 0. For notational ease, let us denote
η1 =s2λ2 − c1s1λ1 − c1
, η2 =s1λ1 − c2s2λ2 − c2
.
July 8, 2005 DRAFT
15
A. Offset Computation
For certain choices of problem parameters, we can have degenerate cases whereP⋆ either
schedulesQ1 in all states, or schedulesQ2 in all states. For the non-degenerate cases, theoffset
is either a point on the line(n2 = 1, n1 ∈ N) or the line(n1 = 1, n2 ∈ N).
From the proof of Lemma 2,P⋆ schedulesQ1 in state (n1, 1), n1 ∈ N if the following
inequality holds:
(s1λ1 − c1)(1 − s1)n1−1 ≥ (s2λ2 − c2).
Note that the left hand side of the inequality if always positive. We have the following cases:
(a) (s2λ2 − c2) < 0: The inequality is trivially satisfied. Thus,P⋆ schedulesQ1 in states
(n1, 1), ∀n1 ∈ N. From Theorem 1 it follows thatP⋆ schedulesQ1 in all states(n1, n2),
n1, n2 ∈ N. Thus, we have a degenrate case.
(b) (s2λ2 − c2) ≥ 0: We can further split this case into two cases:
(i) η1 ≥ 1: The inequality is not satisfied for any choice ofn1. Thus, theoffsetdoes not
lie on the linen1 ∈ N, n2 = 1. Note that this condition is equivalent tos1λ1 < s2λ2.
(ii) η1 < 1: In this case, the largest value ofn1 that satisfies the inequality is given by
n⋆1 =
⌊1 +
log(η1)
log(1 − s1)
⌋. Thus,P⋆ schedulesQ1 for n1 ≤ n⋆
1 andQ2 for n1 > n⋆1
in states(n1, 1). Note that the condition is equivalent tos1λ1 ≥ s2λ2.
Similarly, P⋆ schedulesQ1 in state(1, n2), n2 ∈ N if the following inequality holds:
(s2λ2 − c2)(1 − s2)n2−1 ≤ (s1λ1 − c2).
The possible cases are:
(a) (s1λ1 − c2) < 0: P⋆ schedulesQ2 in all states(n1, n2), n1, n2 ∈ N.
(b) (s1λ1 − c2) ≥ 0: We can split this case further into two cases:
(i) η2 ≥ 1: P⋆ schedulesQ1 in states(1, n2) ∀n2 ∈ N. The offset lies on the line
n1 ∈ N, n2 = 1. Note that this condition is equivalent tos1λ1 ≥ s2λ2.
July 8, 2005 DRAFT
16
(ii) η2 < 1: In states(1, n2), n2 ∈ N, P⋆ schedulesQ2 for n2 < n⋆2 andQ1 for n2 ≥
n⋆2, wheren⋆
2 is given byn⋆2 =
⌈1 +
log(η2)
log(1 − s2)
⌉. This condition is equivalent to
s1λ1 < s2λ2.
We can summarize the above computations as follows:
(a) c1 > s2λ2: P⋆ always schedulesQ1.
(b) c2 > s1λ1: P⋆ always schedulesQ2.
(c) s1λ1 ≥ s2λ2: The offsetis at the point(n⋆1, 1) wheren⋆
1 =
⌊1 +
log(η1)
log(1 − s1)
⌋.
(d) s1λ1 < s2λ2: The offsetis at the point(1, n⋆2), wheren⋆
2 =
⌈1 +
log(η2)
log(1 − s2)
⌉.
We re-emphasize that the aforementioned results hold underthe assumptions(s1λ1 − c1) > 0
and (s2λ2 − c2) > 0.
B. Knee Computation
As discussed earlier, we can have two degenrate cases, namely c1 > s2λ2 whereP⋆ schedules
Q1 in all states andc2 > s1λ2 whereP⋆ schedulesQ2 in all states. There is no need to compute
the knee for these cases and hence we preclude them from consideration in this sub-section.
Now, based on the analysis of the asymptotic behavior ofP⋆ in Section IV, we have two cases,
namely,s1c1 ≥ s2c2 and s1c1 < s2c2. We treat each case separately in the remainder of this
sub-section.
1) s1c1 ≥ s2c2: In this case, the location of thekneeis determined by the zero-crossing of
the sequenceγ(∞, n2). In fact, thekneeis the smallestn2 = n⋆2 for which P⋆ schedulesQ1.
From Lemma 4,γ(∞, n2) is a monotone non-increasing sequence. Thus, it has at most one
zero-crossing, which implies thekneeis unique. From Theorem 1, we know thatP⋆ schedules
Q2 in states(∞, n2) ∀n2 < n⋆2. In particular, from we have from (1)
V (∞, n2) = s2V (∞, 0) + (1 − s2)V (∞, n2 − 1) + c1 + c2 n2 = 2, . . . , n⋆2 − 1.
July 8, 2005 DRAFT
17
Let ψn2 = V (∞, n2), n2 = 2, . . . , n⋆2 − 1. We get the difference equation
ψn2 = s2V (∞, 0) + (1 − s2)ψn2−1 + c1 + c2 n2 = 2, . . . , n⋆2 − 1.
From Lemma 1,V (∞, 0) =c1s1
. Sincec1 < s2λ2, P⋆ schedulesQ2 in the state(∞, 1) and we
getψ1 = V (∞, 1) =c1s1
+ c1 + c2 + (1− s2)λ2. With this initial condition, we recursively solve
the difference equation to obtain
ψn⋆2−1 =
(1 − s2)n⋆
2−1
s2(s2λ2 − c1 − c2) +
c1s1
+c2s2
+c1s2.
Now, by definition ofn⋆2, P
⋆ schedulesQ1 in state(∞, n⋆2. Thus,γ(∞, n⋆
2) ≤ 0. By definition,
γ(∞, n⋆2) = s1α(∞, n⋆
2) − s2β(∞, n⋆2), whereα(∞, n⋆
2) = V (0, n⋆2 − 1) − V (∞, n⋆
2 − 1) and
β(∞, n⋆2) = V (∞, 0)−V (∞, n⋆
2−1). We computeV (0, n⋆2−1) from Lemma 1. UsingV (∞, n⋆
2−
1) = ψn⋆2−1 and settingγ(∞, n⋆
2) ≤ 0 we get
(1 − s2)n⋆
2−1 [(s1c1 − s2c2) + s2(s2λ2 − c1)] ≤ (s1c1 − s2c2).
The kneeis set to the smallest integer value ofn⋆2 that satisfies the above. Hence,
n⋆2 =
1 +log
((s1c1−s2c2)
(s1c1−s2c2)+s2(s2λ2−c1)
)
log(1 − s2)
. (13)
2) s1c1 < s2c2: The computation of thekneeis analogous to the cases1c1 ≥ s2c2. Hence,
we provide only the final results. In this case, theknee is the largestn1 = n⋆1 for which P⋆
schedulesQ1 in state(n1,∞). The location of thekneeis determined by the zero-crossing of
the sequenceγ(n1,∞). In fact, thekneeis set to the largest value ofn⋆1 that satisfies
(1 − s1)n⋆
1−1 [(s2c2 − s1c1) + s1(s1λ1 − c2)] ≤ (s2c2 − s1c1).
Hence,
n⋆1 =
1 +log
((s2c2−s1c1)
(s2c2−s1c1)+s1(s1λ1−c2)
)
log(1 − s1)
. (14)
We provide a numerical example to illustrate the computations in this section.
Numerical Example 1:Let c1 = 2, c2 = 1, λ1 = 103, λ2 = 4 × 103, s1 = 0.8 ands2 = 0.4.
July 8, 2005 DRAFT
18
numex.eps
Fig. 2. Exact switch-over curve and approximate switch-over curve for a typical choice of parameters:λ1 = 103, λ2 = 4×103,
c1 = 2, c2 = 1, s1 = 0.8 ands2 = 0.4
• s1λ1 = 800 < 1600 = s2λ2. Thus, theoffsetis at the point(1, n⋆2) where
n⋆2 = ⌈1 + log(η2)/log(1 − s2)⌉ andη2 = (s1λ1 − c2)/(s2λ2 − c2). We getη2 = 0.4997 and
n⋆2 = 3.
• s1c1 = 1.6 > 0.4 = s2c2. Thus, thekneeis at the point(1, n⋆2) wheren⋆
2 is given by (13).
We getn⋆2 = 14.
The approximate switch-over curve and the exact switch-over (computed by solving the DP) are
depicted in Figure 2.
VI. HEURISTIC SCHEDULING POLICY
So far, we have thoroughly characterized the policyP⋆ which is the optimal packet scheduling
policy for a system with two queues and a single server under static channel conditions. In a
real wireless system, the channel conditions vary randomlywith time, owing to phenomena
like interference from other transmitter or fading due to receiver mobility. Naturally, a packet
scheduling policy must adapt to the varying channel conditions. We have assumed that the
instantaneous channel conditions are perfectly known at the transmitter. This is a reasonable
July 8, 2005 DRAFT
19
assumption when the receivers are moving slowly. With the knowledge of channel conditions,
we assume there is a mappingsi(γi(j);Xi(j), j) : [0,∞); {Xi(j)} → [0, 1] that determines the
probability of successful packet transmission for the queue Qi if it is scheduled in time slotj,
given the instantaneous SNRγi(j) and auxiliary informationXi(j) (modulation scheme, code
rate, spreading factor etc.). Also, we assume that the channel variations are slow enough relative
to the size of the time slot so that the success probability stays constant over a time slot. LetI
denote the index set of all queues at the server and letQj be the index set of non-empty queues
that are possible candidates for scheduling. Thus,Qj = {j : Qj is empty}, Qj ⊆ I. Now, we
propose the following heuristic packet scheduling policyPh.
Repeat:
(a) If Qn = Φ, quit.
(b) If Qn = {k}, k ∈ I then scheduleQk and quit.
(c) SetA = Φ andQ = Qn. Repeat:
(i) If Q = Φ, quit.
(ii) If Q = {k}, k ∈ I then setA = A ∪ {k} andQ = Φ.
(iii) If |Q| ≥ 2, generatek and l uniformly at random such thatk, l ∈ Q andk 6= l.
(iv) Let nk(j) andnl(j) respectively denote the time-to-expiry of the HOL packet ofQk
andQj in time slotj. We have assumed thatsk(γk(j);Xk(j), j) andsl(γl(j);Xl(j), j)
are known. In addition,ck, cl, λk and λl are pre-determined parameters. With these
inputs, use the scheduling policyP⋆ to schedule one of eitherQk or Ql.
(v) If Qk is chosen, setA = A ∪ {k}. If Ql is chosen, setA = A ∪ {k}. In both cases,
setQ = Q\{k, l}.
(d) SetQn = A.
Remarks:A typical instance of the algorithm for a five queue system is illustrated in Figure
3. We note that in step (iv) of (c) above, we can replaceP⋆ by the piece-linear approximation
developed in Section V. We denote the resulting packet scheduling policy by Ph. This policy
July 8, 2005 DRAFT
20
2 3 4 5
2 1
5 43
P⋆
P⋆
4
3
5
P⋆
R
R
5
3 5
3 5 4
P⋆
1
u uu
u u
u u
u u
u
u
u
u u
u
u
u
u
u
Fig. 3. A typical instance of the heuristic scheduling policy Ph. In the figure,R represents random pairing of queues while
P⋆ represents pairwise scheduling using policyP⋆
has computational complexity significantly lower than thatof Ph. As illustrated for a typical
choice of parameters in numerical example 1, the switch-over curves generated byP⋆ and the
piece-wise linear approximation are very close. Thus, we use Ph for performance evaluation in
all our simulation results without any noticeable performance degradation.
A. Computational Complexity
If |I| = K, the pairwise scheduling approach adopted inPh andPh terminates in atO(log2K)
steps. Theith step requires solvingO
(K
2i
)pairwise scheduling problems. Thus, the total number
of pairwise scheduling problems to be solved is upper bounded by K
(1
2+
1
22+ . . .
)= K.
In fact, the number of pairwise scheduling problems to be solved is exactly(K − 1). 3 Using
the approximation developed in Section V, we can solve each pairwise scheduling problem in
O(1) time. Thus, the packet scheduling policyPh hasO(K) computational complexity per time
3Let mk be the number of pairwise scheduling problems that need to besolved for ak queue system. Then,mk = k/2+mk/2
if k is even andmk = (k − 1)/2 + m(k+1)/2 if k is odd. It is easy to see that these equations are satisfied bymk = (k − 1)
∀k ∈ N
July 8, 2005 DRAFT
21
slot. It is interesting to contrast this with the computational complexity of other benchmark
scheduling algorithms. The round-robin scheduler hasO(1) complexity per time slot. All index
based scheduling algorithms, maximum SNR, EDD, M-WLDF etc.haveO(K) complexity per
time slot because they involve computing the maximum of a list of K numbers. PolicyP⋆ has
complexityO(D2) for solving the pairwise scheduling problem, whereD is the maximum of the
times-to-expiry of the two queues under consideration. Thus, policyPh has complexityO(D2K)
per time slot, whereD denotes the maximum time-to-expiry over allQi, i ∈ I.
VII. SIMULATION RESULTS
In this section, we contrast the performance of the proposedscheduling policies with other
benchmark schedulers via link-level simulations. We first introduce the performance metrics used
to evaluate the performance of various scheduling policies.
We use theaverage packet drop rateas a measure of system performance. The packet drop
rate for each queue is computed as the ratio of the number of packet dropped to the total number
of packets processed by the queue (successfully transmitted or dropped) over a sufficiently long
simulation run. We denote this drop rate byPd(k) for the kth queue,Qk. The average drop rate
is simply the average of the drop rates of all queues. We denote it by Pd. Thus, for a system with
K queues,Pd =1
K
K∑
k=1
Pd(k). As discussed in Section I, apart from maximizing some metric of
system performance, it is also desirable for a scheduling policy to possess an element of fairness
so that all queues receive their fair share of service. We use∆ =
(max
1≤k≤KPd(k) − min
1≤k≤KPd(k)
)
as a measure of fairness. Thus, a small value of∆ indicates a higher degree of fairness and
vice-versa.
To decouple the performance of the scheduling policies fromrandomness in the packet arrival
process, we assume that arrivals occur to each queue according to a deterministic process. In
particular, exactly one packet arrives at theith queueQi everyDi > 0 time slots. The inter-
packet deadline associated with this packet isDi, that is, it must be transmitted withinDi slots
of becoming the HOL packet, else it gets dropped. This is a reasonable model for an application
July 8, 2005 DRAFT
22
like live media streaming where packets/frames are generated at a constant rate by the source.
As discussed earlier, the instantaneous channel conditions and the choice of auxiliary parame-
ters like modulation scheme, code rate etc. are captured by amappings(γ; {X}) : [0,∞); {X} →
[0, 1] that represents the probability of successful packet transmissions for instantaneous SNR
γ and choice of auxiliary parameters{X}. We assume thats(·) remains constant over a trans-
mission time interval, a justified assumption for slowly fading channels. In a detailed system
level simulation,s(·) would be specified via actual value interface (AVI) tables [16]. In a real
system,s(·) would typically be gotten from look-up tables available at the BS, or estimated from
measurements at the receiver. For the sake of concreteness,we assume a simple functional form
for s(·), namelys(γ; {X}) = 1 − exp(−δγ), whereδ = δ({X}) is a function of the auxiliary
parameters. We do not expect the relative performance of various scheduling policies to depend
on the choice for the functional form ofs(·). We chose a value ofδ = 0.25 for our simulation
results. To compute the instantaneous SNR, we fix the mean SNRand use a 3-path channel model
(ITU Pedestrian A, 3kmph) with independently Rayleigh faded paths with Doppler spectrum.
We assume that an ideal RAKE receiver is employed at the receiver, so that the energy from all
multipaths is captured completely. All simulation resultsare generated by simulating the system
over 105 time slots. For a slot length of 2ms (HSDPA), the simulation run length is equivalent
to 200 seconds of real time.
A. Scenario A
We consider a system with four queues and deadlinesD1 = 3, D2 = 4, D3 = 5 andD4 = 6
slots respectively. The mean SNR is same for all queues and isvaried from 6dB to 18dB. The
buffering and dropping costs are set equal to1 and 103 respectively for all queues. Figure 4
depicts the fairness v/s average drop rate trade-off for different scheduling policies. The proposed
scheduling policyPh is the most fair for any given average drop rate.
July 8, 2005 DRAFT
23
3456.eps
Fig. 4. Scenario A: Fairness (∆) v/s Average Drop Rate (Pd) trade-off for various scheduling policies for a four queuesystem
with D1 = 3, D2 = 4, D3 = 5, D4 = 6
B. Scenario B
We consider a system with two queues. We fixD2 = 5 and varyD1 from 2 to 5. We set
c1 = c2 = 1, λ2 = 103 and investigate the fairness v/s average drop rate trade-off obtained asλ1
is varied. The mean SNR is get to 10dB for both queues. The trade-off curves forD1 = 2 and
D1 = 3 are depicted in Figure 5. We then simulate the proposed scheduler Ph with the optimal
(most fair) choice ofλ1 for each value ofD1. The performance is contrasted with the EDF and
EXP-rule schedulers in Figure 6. The fairness performance of other schedulers severely degrades
asD1 decreases.
C. Scenario C
We consider a system with two queues. We fixD1 = D2 = 4. The mean SNR for the second
queue is set toγ2 =12dB while the mean SNR for the first queue (γ1) is varied from 6dB to
12dB. We also fixc1 = c2 = 1 andλ2 = 103. Then we study the fairness v/s average drop rate
trade-off obtained asλ1 is varied. The trade-off curves forγ1 =6dB andγ1 =8dB are depicted
July 8, 2005 DRAFT
24
optlam.eps
Fig. 5. Fairness (∆) v/s Average Drop Rate (Pd) trade-off for the scheduling policyPh for two different values ofD1 for a
two queue system, asλ1 is varied
2qdiffdl.eps
Fig. 6. Scenario B: Fairness (∆) v/s Average Drop Rate (Pd) trade-off for various scheduling policiesPh for different values
of D1 for a two queue system
July 8, 2005 DRAFT
25
optlam2.eps
Fig. 7. Fairness (∆) v/s Average Drop Rate (Pd) trade-off for the scheduling policyPh for two different values ofγ1 for a
two queue system, asλ1 is varied
2qdiffsnr.eps
Fig. 8. Scenario C: Fairness (∆) v/s Average Drop Rate (Pd) trade-off for various scheduling policiesPh for different values
of γ1 for a two queue system
July 8, 2005 DRAFT
26
in Figure 7. We then simulate the proposed schedulerPh with the optimal (most fair) choice of
λ1 for each value ofγ1. The performance is contrasted with the EDF and EXP-rule schedulers
in Figure 8. The proposed scheduler provides almost uniformfairness over all choices ofγ1 and
γ2, while the performance of other schedulers degrades asγ1 decreases.
VIII. C ONCLUSIONS
We investigated the problem of downlink packet scheduling in a time-slotted wireless commu-
nication network. We focused on a scenario where packets have associated deadlines, as is the
case in numerous real-time applications like multimedia communication and mobile computing.
We constructed a parsimonious reduction of the scheduling problem and formulated the reduced
problem in a dynamic programming framework. We achieved a tremendous reduction in the
size of the state-space of the DP by using the notion of inter-packet deadlines. We derived key
structural properties of optimal solution to the DP and showed that the optimal policy is of
switch-overtype. We constructed a piece-wise linear approximation to the optimal switch-over
curve. Finally, we proposed a heuristic scheduling policy based on randomization and pairwise
scheduling (using the switch-over policy). We contrasted the performance of the proposed
scheduler with other benchmark scheduling policies via simulations under several different
scenarios and demonstrated that the proposed scheduler yields a significantly better trade-off
between user fairness ans system performance. Future work involves incorporating power control
and adaptive modulation and coding into the scheduling framework.
REFERENCES
[1] H. Holma and A. Toskala, Eds.,WCDMA for UMTS, John Wiley & Sons, 3rd Ed., 2002.
[2] G.L. Stuber,Principles of Mobile Communication, Kluwer, Boston, 2001.
[3] J.G. Proakis,Digital Communications, 4th Ed., McGraw Hill, New York, 2001.
[4] S. Shakkotai, T. Rappaport and P. Karlsson, “Cross-layer design for wireless networks”,IEEE Comm. Mag., vol. 41, no.
10, Oct. 2003, pp. 74-80.
[5] 3GPP, “High Speed Downlink Packet Access (HSDPA); overall description”, 3GPP, Sophia Antipolis, France, Technical
Specification 25.308, Ver. 5.4.0, Release 5, Mar. 2002.
July 8, 2005 DRAFT
27
[6] H. Fattah and C. Leung, “An overview of scheduling algorithms in wireless multimedia networks”,IEEE Wireless Comm.,
vol. 9, no. 5, Oct. 2002, pp. 76-83.
[7] S. Lu, V. Bhargavan and R. Srikant, “Fair scheduling in wireless packet networks”,IEEE/ACM Trans. Networking, vol.
7, no. 4, Aug. 1999, pp. 473-489.
[8] P. Benderet al, “CDMA/HDR: A bandwidth-efficient high-speed wireless data service for nomadic users”,IEEE Comm.
Mag., vol. 38, no. 7, Jul. 2000, pp. 70-77.
[9] S. Shakkottai and R. Srikant, “Scheduling real-time traffic with deadlines over a wireless channel”,Wireless Networks,
vol. 8, no. 1, Jan. 2002, pp. 13-26.
[10] S. Shakkotai and A.L. Stolyar, “Scheduling algorithmsfor a mixture of real-time and non-real-time data”,Proc. of the
17th Intl. Teletraffic Congress (ITC-17), Salvador da Bahia, Brazil, Sept. 2001.
[11] M. Agarwal and A. Puri, “Base station scheduling of requests with fixed deadlines”,Proc. of IEEE INFOCOM, June
23-27, 2002, vol. 2, pp. 487-496.
[12] K.B. Johnsson and D.C. Cox, “An adaptive cross-layer scheduler for improved QoS support of multi-class data services
on wireless systems”,IEEE Journal on Selected Areas in Communications, vol. 23, Issue 2, Feb. 2005, pp. 334-343.
[13] D. Bertsekas,Dynamic Programming and Optimal Control, vol. 1 & 2, 2nd Ed., Athena Scientific, 2000.
[14] J. Walrand,An Introduction to Queueing Networks, Prentice Hall, Englewood Cliffs, NJ, 1988.
[15] J. Huang, R.A. Berry and M.L. Honig, “Wireless scheduling with hybrid ARQ”,
www.ece.northwestern.edu/ rberry/harq.pdf.
[16] S. Hamalainen, P. Slanina, M. Hartman, A. Lappetelainen, H. Holma and O. Salonaho, “A novel interface between link and
system level simulations”,Proc. ACTS Mobile Telecommunications Summit, Aalborg, Denmark, Oct. 1997, pp. 599-604.
IX. A PPENDIX
A. Proof of Lemma 1
When the system is in state(n1, 0), that is,Q2 is empty, there is no scheduling decision to
be made, and the HOL packet ofQ1 is transmitted in successive time slots till it is received
correctly atR1, or dropped due to expiration of its deadline. Thus, we have
V (n1, 0) = s1V (0, 0) + (1 − s1)V (n1 − 1, 0) + c1, n1 > 1
= (1 − s1)V (n1 − 1, 0) + c1, n1 > 1.
With the initial conditionV (1, 0) = c1 + (1 − s1)λ1, we have a simple difference equation of
the formx(n) = ax(n− 1) + b, x(1) = x1, a < 1, the solution to which is given by
x(n) =b
1 − a+ an−1
(x1 −
b
1 − a
). (15)
July 8, 2005 DRAFT
28
Using a = (1 − s1) and b = c1 in (15), we conclude (a). Similarly, if the system is in state
(0, n2), Q1 is empty andQ2 is always scheduled. Thus,
V (0, n2) = (1 − s2)V (0, n2 − 1) + c2, n2 > 1.
Using the initial conditionV (0, 1) = c2 + (1− s2)λ2, and witha = (1− s2) andb = c2 in (15),
we conclude (b).
B. Proof of Lemma 2
From (4) we have,
V (n1, 1) = min{(1 − s1)V (n1 − 1, 0) + λ2, V (n1 − 1, 0) + (1 − s2)λ2} + c1 + c2.
Thus,P⋆ schedulesQ1 in state(n1, 1) if
V (n1 − 1, 0) ≥s2λ2
s1
.
We use Lemma 1 to computeV (n1 − 1, 0) which yields the condition,
(s1λ1 − c1)(1 − s1)(n1−1) ≥ (s2λ2 − c2). (16)
For notational ease, letη1 =s2λ2 − c1s1λ1 − c1
. Sincec1 > s1λ1, we have three possibilities
(a) (s2λ2 − c1) < 0: The inequality in (16) is trivially satisfied. Thus,P⋆ always schedules
Q1 in state(n1, 1) ∀n1 ∈ N.
(b) (s2λ2 − c1) ≥ 0: We further split this into two sub-cases:
(i) η ≥ 1: The inequality in (16) is never satisfied. Thus,P⋆ always schedulesQ2 in
state(n1, 1) ∀n1 ∈ N.
(ii) η < 1: There exists an⋆1 > 1 such thatP⋆ schedulesQ1 in state(n1, 1) for n1 ≤ n⋆
1
andQ2 else. The choice ofn⋆ is given by
n⋆1 =
⌊1 +
log(η)
log(1 − s1)
⌋.
All three (mutually exhaustive) possibilities reduce to one of the three cases stated in the lemma.
Hence, we claim the lemma is true.
July 8, 2005 DRAFT
29
C. Proof of Lemma 4
The proof is through the principle of mathematical induction. We begin with the base case,
that is,n1 = n2 = 2.
1) Base Case:γ(2, 3) ≤ γ(2, 2) ≤ γ(3, 2)
From definitions, we have
γ(2, 3) = s1V (0, 2) − s2V (1, 0) + (s2 − s1)V (1, 2).
γ(2, 2) = s1V (0, 1) − s2V (1, 0) + (s2 − s1)V (1, 1).
γ(3, 2) = s1V (0, 1) − s2V (2, 0) + (s2 − s1)V (2, 1).
We will show thatγ(2, 2) − γ(2, 3) = s1[V (0, 1) − V (0, 2)] + (s2 − s1)[V (1, 1) − V (1, 2)] ≥ 0.
We make three observations:
(a) P⋆ schedulesQ1 in state(1, 1) if s1λ1 ≥ s2λ2.
(b) P⋆ schedulesQ2 in state(1, 2) if s1λ1 ≥ s2[c2 + (1 − s2)λ2].
(c) By assumption,s1λ2 > c2. Thus,λ2 > c2 + (1 − s2)λ2.
Now, we have three cases:
(a) s1λ1 ≥ s2λ2: From observation (a) and (c),P⋆ schedulesQ1 in both states(1, 1) and(1, 2).
Thus,V (1, 1) = (1 − s1)λ1 + λ2 + c1 + c2, V (1, 2) = (1 − s1)λ1 + (1 − s2)λ2 + c1 + 2c2
andγ(2, 2) − γ(2, 3) = s2(1 − s1)(s2λ2 − c2) ≥ 0.
(b) s1λ1 < s2[c2 + (1 − s2)λ2]: From observation (b) and (c),P⋆ schedulesQ2 in both states
(1, 1) and(1, 2). Thus,V (1, 1) = λ1 + (1− s2)λ2 + c1 + c2, V (1, 2) = (1− s2)2λ2 + λ1 +
c1 + (2 − s2)c2 andγ(2, 2) − γ(2, 3) = s2(1 − s2)(s2λ2 − c2) ≥ 0.
(c) s2[c2 +(1−s2)λ2] ≤ s1λ1 < s2λ2: From observation (b) and (c),P⋆ schedulesQ1 in state
(1, 2) andQ2 in state(1, 1). Thus,V (1, 1) = λ1+(1−s2)λ2+c1+c2, V (1, 2) = (1−s1)λ1+
(1− s2)λ2 + c1 + 2c2 andγ(2, 2)− γ(2, 3) = s1(1− s2)(s2λ2 − c2) + (s2 − s1)(s1λ1 − c2).
Using s1λ1 ≥ s2[c2 + (1 − s2)λ2, we getγ(2, 2) − γ(2, 3) ≥ s2(1 − s2)(s2λ2 − c2) ≥ 0.
By similar arguments, we can showγ(2, 2) ≤ γ(3, 2). Hence, the base case is proved.
July 8, 2005 DRAFT
30
2) Inductive Step:We now assume that for somen1, n2 > 2, the following holds:
γ(n1 − 1, n2) ≤ γ(n1 − 1, n2 − 1) ≤ γ(n1, n2 − 1). (17)
We will show that the inductive hypothesis (17) impliesγ(n1, n2+1) ≤ γ(n1, n2) ≤ γ(n1+1, n2).
We will focus on proving the right side of the inequality, that is, γ(n1 + 1, n2) ≥ γ(n1, n2). The
proof for the left side of the inequality is analogous.
By definition, P⋆ schedulesQ1 in state (n1, n2) ∀n1, n2 ∈ N\{1} if γ(n1, n2) ≤ 0 and
schedulesQ2 else. We have the following three cases:
(a) γ(n1, n2−1) ≤ 0: From (17),γ(n1 −1, n2 −1) ≤ 0. Thus,P⋆ schedulesQ1 in both states
(n1 − 1, n2 − 1) and (n1, n2 − 1).
(b) γ(n1 −1, n2 −1) > 0: From (17),γ(n1, n2 −1) > 0. Thus,P⋆ schedulesQ2 in both states
(n1 − 1, n2 − 1) and (n1, n2 − 1).
(c) γ(n1 − 1, n2 − 1) ≤ 0 andγ(n1, n2 − 1) > 0: P⋆ schedulesQ1 in state(n1 − 1, n2 − 1)
andQ2 in state(n1, n2 − 1).
Now,
γ(n1, n2 − 1) − γ(n1 − 1, n2 − 1) =
s2[V (n1 − 2, 0) − V (n1 − 1, 0)] + (s2 − s1)[V (n1 − 1, n2 − 2) − V (n1 − 2, n2 − 2)],
and
γ(n1 + 1, n2) − γ(n1, n2) =
s2[V (n1 − 1, 0) − V (n1, 0)] + (s2 − s1)[V (n1, n2 − 1) − V (n1 − 1, n2 − 1)].
From Lemma 1 we obtain,
V (n1 − 1, 0) − V (n1, 0) = s1V (n1 − 1, 0) − c1
V (n1 − 2, 0) − V (n1 − 1, 0) =1
1 − s1[V (n1 − 1, 0) − V (n1, 0)] .
July 8, 2005 DRAFT
31
For case (a),
V (n1, n2 − 1) = s1α(n1, n2 − 1) + V (n1 − 1, n2 − 2) + c1 + c2
= s1V (0, n2 − 2) + (1 − s1)V (n1 − 1, n2 − 2) + c1 + c2
V (n1 − 1, n2 − 1) = s1α(n1 − 1, n2 − 1) + V (n1 − 2, n2 − 2) + c1 + c2
= s1V (0, n2 − 2) + (1 − s1)V (n1 − 2, n2 − 2) + c1 + c2.
We combine the above equations to get
γ(n1 + 1, n2) − γ(n1, n2) = (1 − s1)[γ(n1, n2 − 1) − γ(n1 − 1, n2 − 1)] ≥ 0.
The inequality follows from the inductive assumption (17).
For case (b),
V (n1, n2 − 1) = s2β(n1, n2 − 1) + V (n1 − 1, n2 − 2) + c1 + c2
= s2V (n1 − 1, 0) + (1 − s2)V (n1 − 1, n2 − 2) + c1 + c2
V (n1 − 1, n2 − 1) = s2β(n1 − 1, n2 − 1) + V (n1 − 2, n2 − 2) + c1 + c2
= s2V (n1 − 2, 0) + (1 − s2)V (n1 − 2, n2 − 2) + c1 + c2.
In this case we can show that
γ(n1 + 1, n2) − γ(n1, n2) = (1 − s2)[γ(n1, n2 − 1) − γ(n1 − 1, n2 − 1)] ≥ 0.
The inequality follows from the inductive assumption.
For case (c),
V (n1, n2 − 1) = s2β(n1, n2 − 1) + V (n1 − 1, n2 − 2) + c1 + c2
= s2V (n1 − 1, 0) + (1 − s2)V (n1 − 1, n2 − 2) + c1 + c2
V (n1 − 1, n2 − 1) = s1α(n1 − 1, n2 − 1) + V (n1 − 2, n2 − 2) + c1 + c2
= s1V (0, n2 − 2) + (1 − s1)V (n1 − 2, n2 − 2) + c1 + c2.
July 8, 2005 DRAFT
32
In this case we can show that
γ(n1 + 1, n2) − γ(n1, n2) = (1 − s2)γ(n1, n2 − 1) − (s2 − s1)γ(n1 − 1, n2 − 1) ≥ 0.
The inequality follows because for case (c),γ(n1, n2−1) > 0 andγ(n1−1, n2−1) ≤ 0. Hence,
we have shownγ(n1 + 1, n2) − γ(n1, n2) > 0 for all three cases. The proof forγ(n1, n2) −
γ(n1, n2 + 1) ≥ 0 is analogous and can be constructed in similar fashion by considering three
mutually exhaustive cases. We omit the details.
We now invoke the principle of mathematical induction to claim that the statement of the
lemma is true for all states(n1, n2), n1, n2 ∈ N\{1}.
D. Proof of Theorem 1
When the system is in state(n1, 0) ∀n1 ∈ N or (0, n2) ∀n2 ∈ N, there is no scheduling
decision to be made andQ1 andQ2 are respectively scheduled. We thus focus on the non-trivial
part of the state space, which is the states(n1, n2) ∀n1, n2 ∈ N, where the scheduler needs to
make a scheduling decision.
Let us focus on the states(n1, n2) ∀n1, n2 ∈ N\{1}. Fix n1 = n′1, say. From Lemma 4,
γ(n′1, n2) is a monotone non-increasing function ofn2. Thus, we have three cases in the state
(n′1, n2) ∀n2 ∈ N\{1}.
(a) γ(n′1, n2) > 0 ∀n2 ∈ N\{1}. Thus,P⋆ schedulesQ2 in the states(n′
1, n2) ∀n2 ∈ N\{1}.
(b) γ(n′1, n2) ≤ 0 ∀n2 ∈ N\{1}. Thus,P⋆ schedulesQ1 in the states(n′
1, n2) ∀n2 ∈ N\{1}.
(c) There is an⋆2 = n⋆
2(n′1) ≥ 1 such thatγ(n′
1, n2) ≤ 0 for n2 ≥ n⋆2 and γ(n′
1, n2) ≤ 0 for
n2 < n⋆2. Thus,P⋆ schedulesQ2 in state(n′
1, n2) for n2 < n⋆2 andQ1 for n2 ≥ n⋆
2.
Hence, there is a switch-over functionφ such thatφ(n′1) = ∞ in case (a),φ(n′
1) ≤ 2 in case
(b) andφ(n′1) = n⋆
2(n′1) in case (c).
From Lemma 4 we know thatγ(n1, n2) is a monotone non-decreasing function ofn1. In
particular,γ(n′1 + 1, n2) ≥ γ(n′
1, n2) ∀n2 ∈ N\{1}. Also, γ(n′1 + 1, n2) is a monotone non-
increasing function ofn2. If case (a) is true,γ(n′1 + 1, n2) > 0 ∀n2 ∈ N\{1}. It is easy to
July 8, 2005 DRAFT
33
show that ifP⋆ schedulesQ2 in state(n1, 2), it necessarily schedulesQ2 in state(n1, 1). Thus,
φ(n′1 + 1) = ∞. If case (b) is true, eitherγ(n′
1 + 1, n2) ≤ 0 ∀n2 ∈ N\{1}, or there is an⋆2 > 1
such thatγ(n′1 +1, n2) > 0 for n2 < n⋆
2 andγ(n′1 +1, n2) ≤ 0 for n2 ≥ n⋆
2. Thus,φ(n′1 +1) ≤ 2
or φ(n′1 +1) = n⋆
2(n′1 +1). The latter is also true for case (c). From the monotone non-increasing
property ofγ(n1, n2) as a function ofn2 we conclude thatn⋆2(n
′1 + 1) ≥ n⋆
2(n′1). Thus, in cases
(a) and (c),φ(n′1 + 1) ≥ φ(n′
1). In case (b), ifP⋆ schedulesQ1 in state(n′1 + 1, 1) then it
necessarily schedulesQ1 in state(n′1, 1), from Lemma 2. Thus,φ(n′
1 + 1) ≥ φ(n′1) in case (b).
The arguments presented so far are true forn1 > 1. For n1 = 1, we invoke Lemma 3. If
P⋆ schedulesQ1 in state (1, n2) ∀n2 ∈ N, then φ(1) = 1. Sinceφ(n1) ≥ 1 by definition,
φ(2) ≥ φ(1). If P⋆ schedulesQ2 in state(1, n2) ∀n2 ∈ N, thenφ(1) = ∞. From Lemma 2,P⋆
schedulesQ2 in state(n1, 1) ∀n1 ∈ N\{1}. Thus,φ(n1) = ∞ ∀n1 ∈ N. Lastly, suppose there
is a n⋆2 > 1 such thatP⋆ schedulesQ2 for n2 < n⋆
2 andQ1 for n2 ≥ n⋆2. Then,φ(1) = n⋆
2. It
can be shown easily thatP⋆ necessarily schedulesQ2 in state(2, n2) if it schedulesQ2 in state
(1, n2). Thus,φ(2) ≥ φ(1).
Hence, for states(n1, n2), ∀n1, n2 ∈ N, we have shown the existence of a non-decreasing
switch-over functionφ such that the optimal policyP⋆ schedulesQ1 in state(n1, n2) if n2 ≥
φ(n1), else it schedulesQ2. We conclude that the optimal policy isP⋆ is a switch-over policy.
E. Proof of Theorem 2
(a) We will use the method of contradiction. Suppose the claim is false. Then, for any choice
of n1 ∈ N, say n′1, there exists an′′
1 > n′1 such thatφ(n′′
1) 6= φ(n′1). It follows from
Theorem 1 thatφ(·) is a monotone non-decreasing function. Thus,φ(n′′1) > φ(n′
1). Also,
φ(n1) ≥ φ(n′′1) > φ(n′
1) ∀n1 ≥ n′′1. We can find such ann′′
1 for every choice ofn1 =
n′1 ∈ N. Hence, by suitably choosing ann′′
1 for eachn′1, we can construct a monotone
strictly increasing sub-sequenceφ(nk). Clearly, this sub-sequence converges to+∞. Since
φ(n) is a monotone non-decreasing sequence, it converges to a limit, and every sub-
sequence ofφ(n) converges to the same limit. It follows that the sequenceφ(n) converges
July 8, 2005 DRAFT
34
to +∞. By definition, φ(n1) is the value ofn2 at whichP⋆ switches over fromQ2 to
Q1 in state(n1, n2) for fixed n1. Alternatively, it is the value ofn2 for which γ(n1, n2)
changes sign from positive to negative, for fixedn1. Now, limn1→∞
φ(n1) = +∞ implies that
limn1→∞
γ(n1, n2) > 0 ∀n2 ∈ N. Thus, limn1→∞
limn2→∞
γ(n1, n2) = γ⋆ > 0. However, from (12)
in Section IV, for the cases1c1 ≥ s2c2 we haveγ⋆ = c2s2
s1
− c1 ≤ 0. This leads us to a
contradiction. We conclude that the claim is indeed true.
(b) We will again employ the method of contradiction. Suppose the claim is false. Then, for any
choice ofn1 ∈ N, sayn′1, there existsn′′
1 > n′1 such thatφ(n′′
1) <∞. From monotonicity
of φ(·), φ(n1) < ∞ ∀n1 ≤ n′′1. Since such a choice ofn′′
1 is possible for eachn′1, we
conclude thatφ(n1) < ∞ ∀n1 ∈ N. By definition, for eachn1 ∈ N, there exists a finite
n2 = φ(n1) such thatP⋆ schedulesQ1 in state(n1, n2) ∀n2 ≥ φ(n1). Alternatively, for
eachn1 ∈ N, γ(n1, n2) changes sign from positive to negative atn2 = φ(n1). In particular,
this is true for limn1→∞
. This implies limn1→∞
limn2→∞
γ(n1, n2) = γ⋆ < 0. However, from (12)
in Section IV, for the cases1c1 > s2c2 we haveγ⋆ = c2 − c1s1
s2> 0. This leads us to a
contradiction. We conclude that the claim is indeed true.
F. Proof of Lemma 5
We define
an2 , limn1→∞
γ(n1, n2) ∀ n2
bn1 , limn2→∞
γ(n1, n2) ∀ n1. (18)
Thus,an2 exists∀ n2, andbn1 exists∀ n1.
We know thatγ(n1, n2) ≥ γ(n1, n2 + 1), therefore{an1} is a decreasing sequence. Thus, it
has a limita⋆. Also, γ(n1 + 1, n2) ≥ γ(n1, n2), therefore{bn2} is an increasing sequence, and
has a limitb⋆. Using (18) we have
limn2→∞
an2 = limn2→∞
limn1→∞
γ(n1, n2) = a⋆
limn1→∞
bn1 = limn1→∞
limn2→∞
γ(n1, n2) = b⋆.
July 8, 2005 DRAFT
35
We have to show thata⋆ = b⋆. We have the following inequalities
limn1→∞
γ(n1, n2) = an2 ≥ a⋆
γ(n1, n2) ≥ bn1 ∀n1
limn1→∞
γ(n1, n2) ≥ limn1→∞
bn1
an2 ≥ b⋆.
The first inequality follows because the{an2} is non-increasing. The second inequality follows
becauseγ(n1, n2) is non-increasing inn2 and has a limit. The third inequality follows directly
from the second, and the fourth follows by definition. Since the above set of inequalities are
true for anyn2, we can conclude thata⋆ ≥ b⋆. Now, since{bn1} is a non-decreasing sequence
andγ(n1, n2) is non-decreasing inn1, we have the following set of inequalities:
limn2→∞
γ(n1, n2) = bn1 ≥ b⋆
γ(n1, n2) ≥ an2 ∀n2
limn2→∞
γ(n1, n2) ≥ limn2→∞
an2
bn1 ≥ a⋆.
Since the above hold for anyn1, we concludeb⋆ ≥ a⋆. Sincea⋆ ≥ b⋆ andb⋆ ≥ a⋆, we conclude
a⋆ = b⋆.
July 8, 2005 DRAFT