See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/3235887 MPLS-Based Satellite Constellation Networks ARTICLE in IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS · MAY 2004 Impact Factor: 3.45 · DOI: 10.1109/JSAC.2004.823406 · Source: DBLP CITATIONS 31 READS 67 3 AUTHORS, INCLUDING: Anton Donner German Aerospace Center (DLR) 54 PUBLICATIONS 123 CITATIONS SEE PROFILE Matteo Berioli Zodiac Aerospace, Wessling, Germany 97 PUBLICATIONS 323 CITATIONS SEE PROFILE Available from: Anton Donner Retrieved on: 04 February 2016
438 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
MPLS-Based Satellite Constellation NetworksAnton Donner, Matteo Berioli, Student Member, IEEE, and Markus Werner, Senior Member, IEEE
Abstract—Nongeostationary satellite constellations with inter-satellite links are a challenge for networking due to their continu-ously changing topology. In order to make maximal use of the net-work’s capacities, special attention has to be paid to routing andtraffic engineering. Multiprotocol label switching (MPLS) as un-derlying protocol is an interesting candidate for this task since itoffers many possibilities to exert influence on traffic flows and sup-ports today’s dominating Internet protocol traffic very well.
This paper describes a general MPLS-based networking conceptfor satellite networks and discusses different scenarios consideringthe particularities and constraints of the dynamic topology. Func-tional elements of MPLS like ingress, egress, or core routers haveto be mapped onto the physical entities of the network and prereq-uisites for traffic engineering are discussed. Routing and reroutingof paths is of key interest since this affects route computation ef-fort and routing performance. Thus, an analytical estimation ofrouting effort is deduced and numerical and simulation results arepresented.
Index Terms—Intersatellite link (ISL), multiprotocol labelswitching (MPLS), nongeostationary orbit (NGSO) satelliteconstellation network, routing effort, traffic engineering.
I. INTRODUCTION
GEOSTATIONARY satellite systems are very well suitedfor broadcast services, but the ever increasing need for
bandwidth of point-to-point and (multi)point-to-multipointapplications have again raised particular interest in broad-band nongeostationary orbit (NGSO) satellite constellations,which offer smaller latency, lower free space loss, true globalcoverage, and better reuse of available ground-space commu-nication frequencies.
Earlier research work on constellations with intersatel-lite links (ISLs) has clearly shown that the asynchronoustransfer mode (ATM) would be an interesting candidate fordynamic satellite network topologies due to its virtual path/vir-tual channel concept and, moreover, as different levels ofquality-of-service (QoS) guarantees are provided. Based onthe connection-orientation paradigm and on a well-specifiedframework of services and traffic classes (including their QoSparameters) supported, reliable and powerful traffic engineeringmethods can be applied in network design [1], [2].
Unfortunately, the more and more dominating connectionlessInternet protocol (IP) does not provide mechanisms for trafficengineering like ATM does, and this imposes to think about aconnection-oriented solution for future IP-based core networks.Multiprotocol label switching (MPLS), an upcoming technique
Manuscript received December 15, 2002; revised July 1, 2003 and November10, 2003. This work was presented in part at the 18th International TeletrafficCongress (ITC-18), Berlin, Germany, September 2003.
Digital Object Identifier 10.1109/JSAC.2004.823406
designed for Internet backbone networks, seems to be an in-teresting candidate to bridge this gap: It allows to adopt newparadigms for conventional IP traffic by decoupling packet for-warding from the information carried in the IP header. This isachieved by distributing routing information to the core routersof the network and assigning short fixed-length ATM-like labelsto IP packets at the ingress point.
MPLS has already been proposed for geostationary earthorbit (GEO) systems with multiple spot beams, mainly as ameans to simplify this complexity of on-board switching [3].However, particularly challenging technology issues arise insuch an environment where huge traffic volumes have to beswitched between several tens or hundreds of input/output portsserving the spot beams.
In contrast to that, the situation is different for low earthorbit (LEO) satellites interconnected by ISLs: A potentially hightraffic load has to be switched between a small number of ports,since the number of ISLs per satellite is strictly limited (typ-ically 4 to 6). This makes constellation networks with ISLsmore similar to terrestrial backbones and, thus, allows to focuson traffic engineering issues related to the dynamic of thesenetworks.
This paper is organized as follows. Section II first recalls rel-evant basics of MPLS and provides a brief introduction to dy-namic satellite constellation networks. A general MPLS-basednetworking concept for such dynamic satellite constellations isthen presented in Section III, and three implementation sce-narios are discussed in more detail, focusing on their respec-tive advantages and drawbacks. As routing performance turnsout to be of key significance in the considered dynamic net-work topologies, a generic analytical estimation of (re)routingeffort in this particular environment is deduced in Section IV,and some numerical and simulation results are presented underreference assumptions. A short summary and an outlook on fu-ture research work concludes the paper.
II. FUNDAMENTALS OF MPLS AND SATELLITE
CONSTELLATION NETWORKS
A. MPLS Key Concepts
The basic idea of MPLS is quite simple: At the ingress pointof an MPLS network, the ingress label edge router (LER),packets are classified according to information carried in thepacket (e.g., source/destination address, service class, etc.) oraccording to network related information (e.g., ingress port), orcombinations of both. A group of packets treated in the sameway is called forwarding equivalence class (FEC). It is alsopossible to bundle a set of FECs and use one single label forthis union. This procedure is known as aggregation.
Then, a unique locally significant fixed-length label is chosenfor packets belonging to a certain FEC and attached to each
DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS 439
packet. Subsequent label switching routers (LSRs) examine thepackets’ labels, replace them with already specified new labels,and forward the packets according to information stored in atable to the next LSR, until the egress point (egress LER) ofthe network is reached. The unidirectional path along which apacket traverses the MPLS domain is called label switched path(LSP).
The forwarding procedure (forwarding plane) is completelydecoupled from the MPLS control plane, which gives serviceproviders a lot of possibilities to influence the network’s be-havior. The control plane itself can be divided into two parts.One part, the label distribution protocol, is responsible for dis-tributing labels to all LSRs along an LSP. Currently, two dif-ferent protocols are defined by the Internet Engineering TaskForce (IETF): 1) RSVP-TE [4] is the traffic engineering exten-sion for MPLS to the resource reservation protocol (RSVP) [5]and 2) CR-LDP [6] is the constraint-based routing label dis-tribution protocol, an extension to label distribution protocol(LDP) [7], which has been newly developed for MPLS. Themain difference between these two label distribution protocolsis the way in which they exchange control information withinthe MPLS domain. RSVP-TE uses a soft state approach withrefresh messages, whereas CR-LDP follows a hard state ap-proach with transmission control protocol (TCP) as underlyingprotocol. An examination of their suitability for a dynamic net-work topology would go beyond the scope of this paper and,furthermore, in February 2003, the IETF decided to discontinuework on CR-LDP [8].
It has to be noted that in a lot of publications label distri-bution protocols are called signaling protocols, but they do nothave much in common with classical signaling protocols likethe ones used in ISDN. Throughout this paper we will make nodifference between the two label distribution protocols as bothcover almost the same functionality.
Although basic traffic engineering can be achieved bymanually defining LSPs (explicit routing), one central goalin networks with fixed nodes is to automatically choose anadequate LSP through the MPLS domain in order to achievemaximal utilization of the network and to meet service classrequirements like delay or guaranteed bandwidth. Therefore,the second part of the control plane consists of mechanismswhich gather network state information and compute routesfor LSPs. An example is OSPF-TE [9], the traffic engineeringextension to open shortest-path first (OSPF) for use withMPLS. Later, in Section III-C, we will see that this approachcontains several drawbacks if used in a satellite constellationand that precalculated explicit routes will be more suitable.
MPLS has been defined to support various data link layersand network layers (cf. Fig. 1). ATM and frame relay can bedirectly used as the MPLS header is mapped onto the existingheader elements; other data link layer protocols need a shimheader to carry the MPLS label information.
Integrated services (IntServ) and differentiated services(DiffServ) are the two philosophies in IP networks to supportQoS requirements. MPLS has been developed to support both:IntServ uses RSVP for making end-to-end reservations, andthe enhancement of RSVP for MPLS, namely RSVP-TE, canbe directly used for setting up LSPs in the MPLS domain [4].
Fig. 1. MPLS between data link layer and network layer [10].
In a DiffServ domain, all IP packets on a link requiring thesame DiffServ behavior constitute a behavior aggregate (BA),which is represented by DiffServ code point (DSCP) attachedto each packet. Mapping of DiffServ behavior aggregates (BAs)entering an MPLS domain onto LSPs can be easily defined; theinterested reader may refer to [11] for further information.
B. Basics of Dynamic Satellite Constellation Networks
In developing an MPLS-based networking concept forsatellite constellation networks, it is first necessary to recalltheir relevant features setting the framework and particularchallenges; among these, the most obvious is the twofold dy-namics of the network: first, variations within the ISL topology,and second, dynamics of the whole ISL network with respectto served ground users.
Earlier studies on the routing in ISL networks of polar ornear-polar star pattern [12] constellations (like Iridium [13])have clearly identified the seam between counter-rotating orbitsand the ON/OFF-switching of its interorbit ISLs as two funda-mental drawbacks for the connection-oriented operation [14].This has stimulated the investigation of moderately inclineddelta pattern [12] constellations employing ISLs [15]. It wasshown that such constellations generally provide the possibilityto set up a number of interorbit ISLs that can be maintainedpermanently with acceptable pointing, acquisition and tracking(PAT) requirements. In particular, pointing investigations haveshown that laser ISL terminals are capable of meeting theazimuth and elevation swivel requirements encountered in suchconstellations with acceptable precision at the necessary speed[16].
Link permanence is especially important for optical ISLs asa time-consuming acquisition of a new neighbor satellite or pe-riodic reacquisition for a switched-off link can be completelyavoided, at least under nominal operating conditions. Moreover,permanence of physical ISLs is in general a highly desirablefeature in the light of real-time connection-oriented services, aspath switching can be completely reduced to such cases where itis inevitable due to handover of the satellites serving the groundusers of considered end-to-end connections.
Celestri [17] was one of the first commercial system pro-posals aiming at the promising combination of a delta patternconstellation and optical ISLs. The Celestri constellation con-sists of 63 satellites evenly distributed in 7 orbit planes, as illus-trated in Fig. 2. Table I lists all relevant constellation and ISLparameters at a glance. Using Celestri as an example, we nowbriefly summarize the major findings and results from a system-atic ISL topology design process [2], referring to Fig. 3.
440 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
Fig. 2. Celestri constellation [17].
TABLE ICELESTRI CONSTELLATION AND ISL PARAMETERS [16], [17]
Fig. 3(a) shows the generic candidate ISLs for a refer-ence satellite in the constellation that may be implementedon principle, and Fig. 3(c) displays the corresponding linklength (and thus delay) variation over one constellation period(roughly 114 min) for the candidate interorbit ISLs. Of theseenvisaged generic ISLs, the two candidates to the after nextorbit, 0–26 and 0–25, are not feasible for the particular Celestriconstellation parameters, due to earth shadowing as indicatedby the dash-dot line representing the respective upper boundfor allowed link length.1 As the curve for the pair 0–9 clearlyreveals, also this generic ISL would not be permanentlyavailable for the same reason. Consequently, choosing only thegeneric interorbit ISL 0–17 to be implemented, we end up withfour physical bidirectional ISLs for each satellite as shown in
1Earth shadowing is here defined as a shortest distance of 150 km betweenthe (optical) link and the earth’s surface, taking into account optical free-spacecommunication conditions.
(a) (b)
(c)
(d)
Fig. 3. ISL topology illustration for the example system Celestri. (a) Genericcandidate ISLs for a reference satellite. (b) Four established ISLs for onesatellite to form a regularly meshed ISL network. (c) Link length variation ofinterorbit ISLs over one constellation period; the dash–dot line represents thebound with respect to earth shadowing. (d) Considered reference ISL topologybased on the Celestri constellation: solid lines, intraorbit ISLs, dashed lines,and interorbit ISLs.
Fig. 3(b). A snapshot of the whole resulting ISL topology forthis case is displayed in Fig. 3(d).2
Note that, due to the link permanence, the shown regular meshtopology does not change during the course of time; only thepointing angles and lengths of interorbit ISLs exhibit periodicchanges with the orbit period. Thus, the interorbit link length isthe only remaining parameter of potential relevance for routingoptions within the ISL subnetwork. However, the major chal-lenge for any connection-oriented networking approach remainsthe inevitable relative movement between the satellite networkand served ground users, which accounts for permanent forcedsatellite handovers.
2Note that the generic ISL topology derived in this way is not the original oneproposed for Celestri, which foresees links to six neighbors.
DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS 441
C. Rerouting in Dynamic Topologies
In satellite constellations and in every MPLS network witha changing topology, rerouting of existing traffic paths is oneof the key issues. The ability to reroute paths and to build re-covery paths does not depend on the signaling protocol, sinceboth label distribution protocols already provide mechanismsfor this purpose:
• fast rerouting;• make-before-break.
Fast rerouting is a technique developed for setting up backuppaths for failure protection of links, nodes, or complete net-work segments. It is very expensive in terms of resource reser-vation and is normally only used in case of very strict reliabilityrequirements.
The idea behind make-before-break is to set up a new LSPin parallel to the still existing old LSP due to topology or trafficvariations. The protocols avoid unnecessary double reservationsof bandwidth on commonly used links, as usually the two linksare used one after the other and not simultaneously. In caseof an explicitly routed LSP, it is of course up to the ingressLER to set up the alternative LSP. Throughout this paper, themake-before-break option is assumed for rerouting in the satel-lite constellation.
Independent of the particular mechanism, however, one keychallenge with rerouting in a connection-oriented environment(with packet sequence typically being preserved) remains theproper alignment of the respective packet streams flowing onthe old and new LSPs, respectively. Such an alignment is es-sential for nondisruptive operation of (mainly real-time) enduser connections served via the LSP. Related solutions havebeen studied in detail for ATM networks. In [18], an align-ment strategy for terrestrial ATM networks is proposed that re-quires duplication of the cell streams during the alignment phaseand thereby guarantees nearly optimal and error-free operation.Similar approaches could certainly be used in an MPLS envi-ronment; this remains an issue for separate research and is notfurther considered in this paper.
III. MPLS NETWORKING CONCEPT FOR
NGSO SATELLITE CONSTELLATIONS
In this section, we bring together the two worlds of 1) MPLSnetworking and 2) satellite constellations employing a dynamicISL topology forming the space-based core network. Wefirst define the appropriate boundaries of the satellite MPLSnetwork, then move on to describing the overall networkingconcept discussing its main functionalities, and finally, presentthree possible implementation scenarios comparing theirrelative advantages and drawbacks. A major distinction be-tween the scenarios is made up by the different locations inthe network, where routing functionality (or intelligence) isphysically established.
A. Network Boundaries
From the previous section, it has become clear that the par-ticular challenge for employing MPLS networking in satellitenetworks is their inherent dynamics. Although, as motivated in
Section II-B, we only consider permanent topologies avoidingthe most unhandy switched ISLs, there are still inherent andfrequent handovers between ground users/stations and servingsatellites which have to be taken into account. Let us considersuch ground users or stations simply as MPLS routers from anetworking perspective. The fact that regular and frequent han-dovers appear between network nodes (routers) is strongly re-lated to the question where to set the boundaries of the MPLSnetwork properly.
As a first option, those satellites serving ground (i.e., due tothe motion of the satellites effectively all satellites at least at acertain time) could be considered as LERs; this would restrictthe MPLS backbone network completely to the ISL space net-work, which has a permanent topology and could thus, on prin-ciple, be operated without stringent LSP rerouting requirements.However, this solution has some serious drawbacks on the otherhand.
LERs, as already explained in Section II-A, constitute ina way the intelligence of the network, they manage the labeldistribution and, in some cases, perform route computations.So placing the LERs “in the sky” means notable on-boardprocessing; we will see in more detail in Section IV howlarge this computational effort can be. Leaving the satellites assimple LSRs to avoid such on-board processing, we must placethe LERs (and the network boundaries) on ground. This causesthe earth-satellite link to fall inside the MPLS network. Thisfirst link becomes very critical for the operation of any LSPusing it, since the link is involved in frequent handovers andthis implies continuous rerouting decisions and computationsfor the LSP.
It should be noted, however, that rerouting has to be per-formed in both cases—either if the LERs are placed on board orif they are placed on ground, respectively—as end-to-end trafficalways flows from ground to ground, but there is a conceptualdifference: In the former case of satellite LERs, the ground sta-tion may trigger a systematic handover from an old to a newserving satellite (LER) by asking to tear down the active LSP(from the old satellite) and sending a new LSP setup request tothe new satellite; consequently, a new phase of LSP activationbegins with its related QoS negotiation and its admission con-trol. In the latter case, when the LERs are placed on ground,approaching a handover they can ask for a rerouting compu-tation, but there is no QoS negotiation nor an admission deci-sion: the LSP was already active and its attributes are alreadyknown, so, if possible, it has to be maintained with the samecharacteristics.
In conclusion, there are no worthwhile advantages to imple-ment LER functionalities on board. Rather two advantages ofhaving LERs placed on ground are dominating: first, there isno need to restart a QoS negotiation or admission control forrerouting of an LSP due to satellite handover; second, expensiveand complex on-board processing for advanced routing func-tionality is avoided.
B. Overall Concept and Decomposition of Functionalities
Developing a tailored MPLS networking concept for satelliteconstellations implies also to consider issues related to various
442 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
Fig. 4. Functional blocks and their interrelation for MPLS-based networkingin dynamic satellite constellations.
protocols involved in MPLS. To this end it is helpful to sepa-rate and break down the functionalities starting from top levelas illustrated in Fig. 4. All relevant MPLS functionalities in dy-namic satellite networks can be summarized in four main func-tional blocks:
1) Network State: including in particular monitoring andstate information distribution;
2) User Behavior/QoS Requirements: making up the de-mand;
3) Route Computation: as the key block comprising “in-telligence” in terms of traffic engineering, adaptiveness,and/or optimization;
4) LSP Rerouting/Creation/Release: establishing the resultof “route computation” in the network.
The distribution of network state information is usually per-formed through link state routing protocols such as OSPF orintermediate system to intermediate system (IS-IS), that canspread various link attributes.
In the ISL network scenario, relevant state information com-prises link availability, capacity, available bandwidth after re-source allocation, and delay (if not calculated on demand, orprecalculated and kept in memory, as the topology is period-ically deterministic). This information must be provided to allnodes that perform route computation. Another important infor-mation to be spread is the state of active LSPs to perform properrerouting in case a traversed link fails or when a handover has tobe performed because of the regular motion of the constellation.
“Route computation” deals with QoS negotiation, admissioncontrol, and traffic engineering issues in reaction to incoming
LSP requests, constituting quite a complex set of procedures. AnLSP request is formulated including particular constraints thathave to be respected by the computed path. This module evalu-ates all incoming requests, checks whether the satellite networkis able to host the requested LSPs, and gives the authorization toset up the paths. For the scope of this work, we can imagine thatthe whole satellite network belongs to only one service provider,which can manage this QoS negotiation as preferred. So wecan assume, for example, that requests are formulated in termsof source and destination LERs, maximum tolerated delay, andminimum required bandwidth along the path.
Once an LSP request is accepted, its negotiated QoS attributesare typically passed to a constraint-based routing module, whichtakes care of calculating the path accordingly. When a path isreleased, there is no need for any computation, but this infor-mation is simply spread and the relative labels are set free. Con-straint-based routing is usually performed as optimization withrespect to a scalar metric, and may be implemented using oneof several proven algorithms where the choice depends on theparticular set of optimization metric and constraints.
In a traffic adaptive scheme, routes are typically chosen insuch a way that traffic load is evenly distributed in the satellitenetwork, LSPs are routed around network link failures andbottlenecks, and congestion is avoided as far as possible. Thereare several algorithms that perform such computations, payingattention to different aspects reflected in respective metrics.Some algorithms try to find always the shortest path for eachLSP (like the commonly used Dijkstra shortest-path algorithm)in terms of either number of hops or generalized link cost.Other algorithms try to maximize the probability to acceptfuture routing demands; one candidate of the latter, minimuminterference routing algorithm (MIRA) [19], has already beentested for MPLS. Finally, earlier work on ATM-based satelliteconstellation networks has focused on algorithms that try toexploit the periodicity of the ISL topology in order to minimizethe number of path reroutings within the ISL subnetwork [14],[20].
An important aspect that should be discussed in this contextis the frequency of route recomputations (see Section IV). Oneoption is that a route is only calculated once after a setup requestand each LSP would then stick to this route until a handover be-tween ground and serving satellite causes a recomputation. Al-ternatively, routes could be recomputed periodically, which mayprove useful in the considered dynamic scenario with continu-ously changing network state. Better routes may become avail-able for already existing paths, resulting in a network perfor-mance gain. However, such an approach would also imply toreroute paths not only because of satellite handover or link fail-ures, but also to improve utilization or adapt to priorities. So anyQoS or performance gain must be thoroughly traded off versusan increase in both computational effort and signaling load dueto label distribution. In particular, it should be evaluated whethercomputationally heavy periodic routing updates are worthwhile,and whether optimizing already existing LSPs does not causetoo many rerouting procedures on the other hand.
In the remainder of this section, we now focus on the questionwhere routing functionality should be physically implemented
DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS 443
in the network, presenting three respective candidate scenariosfor implementation. Fig. 5 provides an illustrative and simpli-fied view on their key characteristics at a glance.
C. Scenario 1: Distributed Routing and LSP Management
A direct adaption of the terrestrial situation to the satellitenetwork is shown in Fig. 5(a). All LERs on ground and the LSRson-board the satellites maintain identical link state databases(LSDBs) representing the entire network’s state, and routes arecomputed at the ingress points of the network. It is up to theingress LER on ground to initiate a setup of and switch overto an alternative LSP according to the information stored in itsLSDB and, therefore, it is crucial to receive continuous updatesfor the LSDB.
In an IP network with fixed nodes and no traffic engineeringrequirements, OSPF version 2 [21] is used as protocol todistribute network information and to synchronize the linkstate advertisements (LSAs) in the LSDBs of the nodes. Thisbasic OSPF version judges links between nodes accordingto a single dimensionless metric and each router constructsa tree of shortest paths with itself as root according to thismetric. LSDBs are updated periodically and due to changesin the network topology, but it is assumed that new networkinformation is distributed very quickly in order to avoid routingerrors or loops. It is clear that this solution is not suitable forMPLS traffic engineering and, therefore, enhancements areunder study in the IETF.
OSPF-TE uses opaque LSAs [22] and distributes much moreinformation than standard OSPF can do. Besides a traffic engi-neering metric the routers know also about maximum (reserv-able/unreserved) bandwidth and can include these constraints intheir path computation. In order to avoid an excessive floodingof the network with LSAs, a link attribute (e.g., bandwidth,which can vary quickly) may have to cross a certain thresholdbefore an update is sent, but all other link attributes are dis-tributed as well. Ongoing further developments try to avoid thisunnecessary traffic by distributing only incremental updates ofsingle attributes without sending the complete LSA, but againthe whole network has to be flooded.
In spite of the improvements of OSPF for use with MPLS, adirect adaption of the terrestrial scenario to the satellite networkis abundantly questionable. The nodes are not fixed and there is acontinuously changing delay of the interorbit links which has tobe monitored and announced to all LSDBs. Moreover, the syn-chronization of the LSDBs suffers from this changing interorbitsatellite distance and the high signal propagation delay causedby the earth enclosing size of the topology, too. It would be pos-sible to avoid OSPF traffic related to a change in link delay byprecalculation in every node as the constellation moves in a de-terministic manner, but for reasonable traffic engineering it isstill necessary to distribute link state information.
The second drawback of this scenario is its inability to sen-sibly modify existing traffic flows. Although MPLS offers thepossibility that traffic with high priority may suppress trafficwith less priority, this behavior is not desirable as the ingressnode of the traffic with low priority has to be notified about this“link interruption” via a link failure message or an update of itsLSDB and it has to set up a complete new path through the net-work on less utilized links again.
D. Scenario 2: Centralized Routing—Distributed LSPManagement
A more efficient traffic engineering can be performed onlywith a global view of the network. For this, it is necessary thateach ingress LER trying to access the network sends the trafficparameters (traffic category, bandwidth, etc.) to a central data-base, which has total knowledge of the network state and com-putes an optimum route for the incoming request. Then, thecentral database answers with a message containing the explicitroute (ER) for an LSP which the ingress LER has to create itself.In parallel, the central LSDB may send new ERs for existing
444 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
LSPs to other ingress LERs in order to trigger handover eventsor to reroute traffic flows for a better network utilization.
This approach [see Fig. 5(b)] has two major advantages: first,satellites with their typical constraints in computing power andmemory do not have to maintain LSDBs and perform routecalculations, and second, the network utilization is increaseddrastically [23]. The drawbacks are the necessity of a new pro-tocol exchanging information between ingress LER and centralLSDB and a more or less accurate knowledge of the ground sta-tions’ coordinates. In scenario 1, the ground stations only needinformation about visibility and distance of satellites for deter-mining alternative LSPs and the time to switch to the new path.Scenario 2, however, offers two possibilities regarding the timeof rerouting: either the central LSDB offers one or several alter-natives for ERs and the decision when to start the rerouting iscompletely up to the ground station (like in scenario 1), whichthen of course has to inform the central LSDB immediatelyabout any action, or the ground station has to take the new routedirectly after reception of the ER from the central LSDB. Theformer option does not require detailed position informationand is suitable for satellite constellations with several visiblesatellites at the same time (satellite diversity), out of which thebest one is chosen, and the latter option is more appropriate fora small number of visible satellites, but then the LSDB needsvery accurate information about the ground stations’ locationsto avoid routing errors.
E. Scenario 3: Centralized Routing and LSP Management
Further development of the centrally triggered LSP reroutingleads to the absolute control scenario shown in Fig. 5(c). Here,the ingress points of the network do not set up LSPs any more;all nodes of the network get their tables for label swapping di-rectly from the central database via logical links. Any decisionabout traffic engineering driven rerouting or handover events isup to the LSDB, but this approach has only little remaining com-monalities with terrestrial use of MPLS, including for instancethe label swapping mechanism.
One advantage of this approach is the faster installationof LSPs. The central LSDB distributes label swapping tablesamong the satellites directly after the reception of a setuprequest from one of the LERs, and of course due to handoveror rerouting events. Then, it sends an acknowledgment back tothe origin of the connection request and this may immediatelystart to use the already existing LSP without having to set upone itself.
In case of handover or rerouting events, the procedure hasto be carefully considered with respect to synchronization re-quirements: all updated tables must start to be used at the sametime. A possible solution is to update all tables along the newLSP with the ingress LER being the last node updated. A majordrawback of this scenario is the design of a new signaling pro-tocol to distribute labels among the LSRs.
Both centralized scenarios (2 and 3) present scalability prob-lems with respect to the number of LSPs: the central station mayonly be able to handle up to a limited number of them. This is a
crucial point and is, therefore, (implicitly) addressed in the fol-lowing when we estimate the routing/rerouting effort requiredto maintain a certain number of LSPs.
IV. ESTIMATION OF ROUTE COMPUTATION EFFORT
The previous section has clearly indicated the crucial signif-icance of (re)routing performance in the considered dynamicscenario. Consequently, the effort which is spent in route com-putations for both LSP setup and rerouting is a key performanceparameter for the operation of a satellite MPLS network. Routecomputation effort may be measured in several ways and at dif-ferent levels (e.g., related amount of signaling, calls of the basicrouting algorithm, elementary processor operations, etc.). Here,we are interested in a simple, yet meaningful, general metric.For the scope of this paper, this is the sheer number of LSPsetup and rerouting occurrences, and more specifically, just its(long-term) time average value for the whole network. This se-lection is mainly motivated by two facts: first, for any more so-phisticated metric this number would certainly remain an impor-tant multiplicative factor; second, one single average value is aquick and intuitive way to perform qualitative case studies—asneeded at this initial stage of our research—and to “compare ata glance” complete global traffic scenarios and/or different con-stellations.
In the following, we first develop a generic analytical estima-tion approach. Subsequently, a simple numerical example is dis-cussed to derive qualitative, as well as first quantitative conclu-sions that are indicative for the ongoing research work. Finally, asimulation framework for discrete-event simulation and perfor-mance evaluation of the MPLS networking and LSP (re)routingscenario is presented, and simulation results are compared withthe analytically derived figures.
A. Analytical Model
Before setting up and running extensive simulations of thesatellite MPLS network, we are keen on developing an analyt-ical model that allows to estimate the route computation effortunder some simplifying assumptions, yet capturing the “live’”network conditions in a sufficiently realistic way. A well-provenapproach for such cases has often been a queueing system mod-eling, where the key model parameters can be adapted to anynumerical real-world situation in a tractable way.
As concluded in Section III-A, we assume that LERs are lo-cated on ground and that all satellites exclusively act as LSRs.Taking a queueing modeling approach, we further assume thateach ground LER receives requests to create LSPs according toa Poisson arrival process with a mean arrival rate . Thismeans equivalently that the system has the property to be mem-oryless and that the interarrival times of the requests are nega-tive exponentially distributed [24]. In such a system, if a groundLER remains inside the footprint of one particular satellite (seeFig. 6) for a time , the probability that in this time interval
LSP requests are generated from the LER is
(1)
It has now to be considered that inside one satellite footprintthere can be several ground LERs at given instants of time. Fig. 7
DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS 445
Fig. 6. LERs inside the footprints of moving satellites.
Fig. 7. Satellite footprint moves across several LERs on ground.
illustrates the simplified geometry of an example situation andthe respective time-varying number of LERs served by the con-sidered satellite. Each of the LERs would in reality set up andtear down its own LSPs, and in particular would obviouslybe strictly related to the concrete position of the LER with re-spect to the edge of the footprint and direction of satellite move-ment, so in essence a deterministic value in each single case.However, with our stochastic modeling approach, we are aimingat an average performance value over time and over the wholenetwork anyway. For the sake of notation simplicity, we furtherconsider as the effective average arrival rate and as the ef-fective average remaining time of the LSPs served by one satel-lite, respectively, no matter if they actually originate from oneor more LERs. One could alternatively assume that all LSPs ina satellite always come from one fictive ground LER only.
We now assume that the duration of an LSP itself (until itexpires, not affected by handover/rerouting) is negative expo-nentially distributed with an average LSP duration of .
Under the further assumption that all LSP requests can be ac-cepted, the above characteristics hold for active or establishedLSPs in the satellite MPLS network. Thus, our system model isan queueing system: the service time (LSP duration)does not depend on the number of users (active LSPs) insidethe system. The rate at which LSPs are torn down in one satel-lite, on the contrary, directly depends on the number of activeLSPs, and it can be shown [24] that the average number of users
in the considered queueing system is , establishing equi-librium operation. In our case, we can, thus, state the averagenumber of active LSPs in one satellite as
(2)
This is an important first result for our satellite network model.So far, we have not taken into account that in the considered
scenario satellites typically serve ground LERs at all only duringa certain share of time , namely when they are overpopulated areas, whereas they are “idle” over deserted regionsand oceans. Or equivalently, the average number of satellitesserving ground stations is
(3)
Now, in the whole constellation there areunordered origin-destination (OD) satellite pairs. As we arelooking at unidirectional LSPs established between pairs ofsatellites, we would start to consider ordered ODpairs with potentially established LSPs. Following our earlierterminology and considerations, the average number of ordered“active” OD pairs would amount to and, thus,
. In other words, in the average, each of theactive satellite shares its active paths with partnersatellites. Thus, the average number of active LSPs for a givenOD pair is
(4)
With being the average handover rate—for both LERsand LSPs according to our earlier assumptions—and assumingcompletely uncorrelated handover occurrence in the corre-sponding source and destination satellites, the handover ratefor one single LSP is simply the sum of the handover rates ofits source and destination satellites
(5)
and with (4) the LSP handover rate for each OD pair becomes
(6)
Multiplication of this value with the average number of or-dered “active” OD pairs gives us the average LSP handover ratefor the whole network
(7)
Finally, the transition back to route computation effort in thenetwork is straightforward due to the chosen simple metric forthe latter, namely, average number of (re)routing occurrencesper time unit. The rerouting effort per time unit is simplyidentical with the average LSP handover rate in the networkas given in (7), and the routing effort related to setup of new
446 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
LSPs per time unit, is just the product of their arrival rateper satellite with the average number of active satellites .Thus, the total network route computation effort becomes
(8)
B. Qualitative and Numerical Discussion
In the following, we assume and as fixed values re-flecting the dynamic constellation considered. and are thetwo parameters of interest. First of all, increases with both
and , i.e., routing effort increases when LSP requests comemore often and when requested LSP durations get longer.
More specifically, the routing effort related to setup of newLSPs, , of course, depends only on their arrival rate ,whereas the rerouting effort , on the other hand, depends onthe average number of active LSPs , because they remain inthe constellation during handovers.
Let us also state the following two indicative results from (8),concerning the relative contributions from LSP setup and LSPrerouting:
• the rerouting effort is smaller than the setup effort if theaverage LSP duration remains below half the average timetill handover
(9)
• the rerouting effort becomes completely dominating (andthe setup effort negligible) if the average LSP duration ismuch larger than half the average time till handover
(10)
Now, consider that realistic values of in satellite constella-tions are in the order of tens of minutes, whereas LSPs in MPLSnetworks certainly last much longer if applied in typical fashion.Thus, in satellite constellation networks employing MPLS, thererouting effort can be expected to be dominating—unlike interrestrial MPLS networks. This fact raises the importance ofresearch into highly efficient LSP rerouting mechanisms for thesatellite scenario.
Regarding the scalability problem of a centralized approach,from (10), one can see that the rerouting effort increases linearlywith the number of active LSPs, and a similar behavior can beexpected for the amount of signaling.
Let us finally adopt some reference values to perform a smallnumerical case study. We fix the constellation related values
and , and let and vary as free parameters. For a givenconstellation, we want to see what happens if incoming trafficchanges its characteristics, namely the average rate of incomingLSP requests , and the average requested LSP duration .
We set the following:
• (for the Celestri example constellation);• (roughly reflecting a reasonable share of land-
masses with respect to the earth’s surface).The parameter can be estimated in the following way. Theduration in which a ground station is passed by a satellite foot-print is determined by the minimal elevation , which is 16for the Celestri example, the orbit altitude , and of course the
TABLE IINETWORK ROUTE COMPUTATION EFFORT R FOR SELECTED COMBINATIONS
OF � AND �. THE NUMBER OF ACTIVE LSPS PER SATELLITE
IS CONSTANT P = �=� = 600
length of the track of the ground station in the satellite foot-print (a small circle on the sphere), which can range from thefootprint’s diameter to very small values next to the edge of thefootprint. Using the (one-sided) earth central angle , which isdetermined by
(11)
with the mean equatorial earth radius km, the meanduration a ground station stays inside a satellite footprint isgiven by
(12)
with the orbit period . Assuming that the terminal on groundalways chooses the nearest satellite (i.e., the satellite with max-imal elevation) and uses it until it falls below the horizon (i.e.,below ), the mean time a ground station remains attachedto the same satellite, can be estimated to . A nu-merical integration of (12) with the Celestri parameters gives
s.The numerical results are summarized in Table II and in
Fig. 8. They confirm the above qualitative considerations. Inparticular, Fig. 8(b) illustrates how the route computation effortasymptotically approaches the value set in (10) when reroutingbecomes dominating.
C. Simulation Approach and Results
In order to validate the presented results, a simulation envi-ronment has been set up based on the network simulator ns-2.3
Values for the Celestri constellation were chosen according toTable I, and eight ground stations were placed randomly nextto big cities all over the world. LSP requests were generated atdifferent Poisson rates and the average LSP duration wasset accordingly in order to achieve a mean total number of ac-tive LSPs in the network equal to .
In the simulation environment LSP requests are not generatedin every ground station, nor in every satellite, but in the overallsystem. Thus, we have that in the entire constellation LSPsper second are generated, and so corresponds to in ourtheoretical discussion. This choice avoids the estimation ofwhich is a very system-dependent value and which has no bigimportance for our scope. Thus, using (8) and ,we obtain for the comparison with our simulation
(13)
Each ground station chooses its serving satellite according tothe description in Section IV-B, i.e., it initially uses the satellite
3http://www.isi.edu/nsnam
DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS 447
(a)
(b)
Fig. 8. Route computation effort (R ). The circles denote the values in thefirst three columns of Table II. (a) “Full” range, up to large numbers of activeLSPs P . (b) Focus on the considered numerical example of P = 600.
with maximal elevation. The route in the space segment betweenorigin and destination satellite is calculated with a Dijkstra al-gorithm, and this route is only recalculated if one of the twoserving satellites falls below the minimal elevation and a han-dover has to be performed.
The simulated results in Fig. 9 follow the theoretical expec-tations very well. It is not surprising that the simulated gen-eration rate of new LSP setups coincides with the prediction,since a plain Poisson process with no particular modificationshas been used. On the other hand, the simulated rate of reroutingevents shows especially for short LSP durations discrepancies tothe expected behavior. The reason is that short-term LSPs havea lower probability to be rerouted since their lifetime is morelikely to fall inside an interval with no handovers, which reducesdrastically the rerouting rate , and this was not consideredin the analytical model.
For larger values of the LSP mean holding time the simu-lated samples cross the prediction (the LSP lifetime does not fitany more into intervals with no handovers) and remain slightlyabove the theoretical result. As already stated in Section II-B,the advantage of inclined Walker constellations is the permanent
Fig. 9. Comparison between theoretical and simulation results.
network topology in the space segment, but this does not equiv-alently mean that each ground station has a similar view of theserving satellites. Users in the very northern or southern hemi-sphere (latitude greater than the inclination of the satellites) willalways have a shorter mean visibility of their serving satellitesthan a user near to the equator, who can experience a visibilitytime up to the equivalent to half a footprint diameter. In otherwords, the mean duration a ground station near the poles staysinside a satellite footprint is smaller than for a terminal near tothe equator, and this causes a higher handover rate. In our sim-ulation, we did not consider terminals in the Antarctic or in theregion around the north pole (edge of constellation coverage)and, thus, the simulated does not show a considerable de-viation from the theoretical result.
V. CONCLUSION
In this paper, we have developed an MPLS networkingconcept for use with NGSO satellite constellations. Offeringrerouting capabilities, MPLS is well suited for use in dynamictopologies. We have also shown that a global view of thenetwork in terms of a central LSDB with central route calcu-lation is necessary for operating the network with guaranteedQoS requirements, which benefits a lot from the periodicityand regularity of Walker delta constellations. Scenarios 2 and3 covered this topic in many details, but did not discuss theproblem of signaling traffic from this single control station tothe network. Other implications like signaling delay and theperformance of the label distribution protocols in the satelliteenvironment have to be addressed in future work, as well assolutions like optimization of the control station’s location orthe use of multiple synchronized control stations.
As routing performance is particularly crucial in the consid-ered dynamic network topology, a generic analytical estimationof route computation effort has been developed. The approachallows to derive qualitative, as well as initial quantitative con-clusions that are indicative for the ongoing research work. It hasbeen demonstrated as well that the LSP rerouting effort tends tobe dominating over the LSP setup effort for long LSP lifetimes
448 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004
and that the simple analytical estimation coincides with our sim-ulation results very well.
Future research work will therefore focus on efficiency andoptimization of rerouting algorithms and performance. Otherpossible research directions are the investigation of (re)routingalgorithms for multicast, traffic load balancing in the constella-tion and for a stateless QoS support over MPLS (such as Diff-Serv over MPLS).
REFERENCES
[1] E. Lutz, M. Werner, and A. Jahn, Satellite Systems for Personal andBroadband Communications, 1st ed. New York: Springer-Verlag,2000.
[2] M. Werner, J. Frings, F. Wauquiez, and G. Maral, “Topological design,routing and capacity dimensioning for ISL networks in broadband LEOsatellite systems,” Int. J. Sat. Commun., vol. 19, no. 6, pp. 499–527,Nov./Dec. 2001.
[3] T. Ors and C. Rosenberg, “Providing IP QoS over GEO satellite sys-tems using MPLS,” Int. J. Sat. Commun., vol. 19, no. 7, pp. 443–461,Sept./Oct. 2001.
[4] D. Awduche, M. Networks, L. Berger, T. L. D. Gan, V. Srinivasan, andG. Swallow, “RSVP-TE: Extensions to RSVP for LSP tunnels,” IETF,RFC 3209, Dec. 2001.
[5] R. Braden, L. Zhang, S. Berson, S. Herzog, and S. Jamin, “Resourcereservation protocol (RSVP)—Version 1, Functional Specification,”IETF, RFC 2205, Sept. 1997.
[6] B. Jamoussi, L. Andersson, R. Callon, R. Dantu, L. Wu, P. Doolan, T.Worster, N. Feldman, A. Fredette, M. Girish, E. Gray, J. Heinanen, T.Kilty, and A. Malis, “Constraint-based LSP setup using LDP,” IETF,RFC 3212, Jan. 2002.
[7] L. Andersson, P. Doolan, N. Feldman, A. Fredette, and B. Thomas, “LDPspecification,” IETF, RFC 3036, Jan. 2001.
[8] L. Andersson and G. Swallow, “The multiprotocol label switching(MPLS) working group decision on MPLS signaling protocols,” IETF,RFC 3468, Feb. 2003.
[9] D. Katz, D. Yeung, and K. Kompella, “Traffic engineering extensionsto OSPF, version 2,” Internet Draft draft-katz-yeung-ospf-traffic-09.txt,work in progress, Oct. 2002.
[10] B. S. Davie and Y. Rekhter, MPLS: Technology and Applications. NewYork: Academic, 2000.
[11] F. Le Faucheur, L. Wu, B. Davie, S. Davari, P. Vaananen, R. Krishnan, P.Cheval, and J. Heinanen, “Multiprotocol label switching (MPLS) sup-port of differentiated services,” IETF, RFC 3270, May 2002.
[12] J. G. Walker, “Circular orbit patterns providing continuous whole earthcoverage,” Royal Aircraft Establishment, U.K., Tech. Rep. 70 211 (UDC629.195:521.6), Nov. 1970.
[13] J. Hutcheson and M. Laurin, “Network flexibility of the IRIDIUMglobal mobile satellite system,” in Proc. 4th Int. Mobile Satellite Conf.(IMSC’95), Ottawa, ON, Canada, June 1995, pp. 503–507.
[14] M. Werner, C. Delucchi, H.-J. Vögel, G. Maral, and J.-J. De Ridder,“ATM-based routing in LEO/MEO satellite networks with intersatellitelinks,” IEEE J. Select. Areas Commun., vol. 15, pp. 69–82, Jan. 1997.
[15] M. Werner, A. Jahn, E. Lutz, and A. Böttcher, “Analysis of system pa-rameters for LEO/ICO-satellite communication networks,” IEEE J. Se-lect. Areas Commun., vol. 13, pp. 371–381, Feb. 1995.
[16] T. Dreischer, H. Kellermeier, E. Fischer, and B. Wandernoth, “Advancedminiature optical terminal family for inter-satellite links in space com-munication networks,” in Proc. 4th European Conf. Satellite Communi-cations (ECSC 4), Rome, Italy, Nov. 1997, pp. 73–78.
[17] Motorola Global Communications, Inc., “Application for authority toconstruct, launch and operate the Celestri multimedia LEO system,”FCC filing, Washington, D.C., June 1997.
[18] B. Edmaier, J. Eberspächer, W. Fischer, and A. Klug, “Alignment serverfor hitless path-switching in ATM networks,” in Proc. XV Int. SwitchingSymp. (ISS’95), Berlin, Germany, Apr. 1995, pp. 403–407.
[19] K. Kar, M. Kodialam, and T. V. Lakshman, “Minimum interferencerouting of bandwidth guaranteed tunnels with MPLS traffic engineeringapplications,” IEEE J. Select. Areas Commun., vol. 18, pp. 2566–2579,Dec. 2000.
[20] M. Werner, “A dynamic routing concept for ATM-based satellite per-sonal communication networks,” IEEE J. Select. Areas Commun., vol.15, no. 8, pp. 1636–1648, Oct. 1997.
[21] J. Moy, “OSPF Version 2,” IETF, RFC 2328, Apr. 1998.[22] R. Culton, “The OSPF Opaque LSA Option,” IETF, RFC 2370, Apr.
1998.[23] G. Haßlinger and S. Schnitter, “How service providers can profit from
traffic engineering,” in Proc. EURESCOM Summit 2002, Heidelberg,Germany, Oct. 2002.
[24] L. Kleinrock, Queueing Systems. New York: Wiley, 1976.
Anton Donner received the Dipl.-Ing. degree in elec-trical engineering from Munich Technical University,Munich, Germany, in 1999.
From 1999 to 2000, he has been working as aResearch Assistant at the Institute of Communica-tions Engineering, Munich Technical University onadaptive channel coding systems for progressivelysource coded data. Since 2000, he has been withthe German Aerospace Center (DLR), Oberpfaffen-hofen, Germany, in the Digital Networks Group ofthe Institute of Communications and Navigation.
His main research activities are satellite-based reliable multicast solutions, andnetworking in nongeostationary satellite systems, with focus on signaling androuting issues in dynamic intersatellite link networks.
Matteo Berioli (S’03) received the Laurea degree inelectronic engineering (magna cum laude), from theUniversity of Perugia, Perugia, Italy, in 2001, wherehe is currently working toward the Ph.D. degree ininformation and electronic engineering.
He has worked for the Italian Army as SecondLieutenant in the telecommunication field. Since2002, he has been with the German Aerospace Center(DLR), Oberpfaffenhofen, Germany, in the DigitalNetworks Group of the Institute of Communicationsand Navigation. His main research activities include
traffic engineering and routing on IP-based dynamic networks, with a focus onMPLS and nongeostationary satellite systems. He also produced some work onIP mobility and security. He is currently involved in two IST projects: FIFTHand Wireless Cabin.
Markus Werner (M’92–SM’01) received theDipl.-Ing. degree from Darmstadt Technical Univer-sity, Darmstadt, Germany, in 1991, and the Ph.D.degree from Munich Technical University, Munich,Germany, in 2002, both in electrical engineering.
Since 1991, he has been with the Institute of Com-munications and Navigation, German AerospaceCenter (DLR), Oberpfaffenhofen, Germany, asResearch Scientist, Project Manager, and GroupLeader. Since 2002, he is also Managing Directorof TriaGnoSys GmbH, Wessling, Germany, a
consulting company for satellite and aeronautical communications. His projectexperience includes several national and ESA studies and various projects inthe framework of European ACTS, IST, and COST research programs. Hismain research activities are in the areas of multiservice traffic engineeringand capacity dimensioning for satellite systems in general, and networkingof nongeostationary satellite systems in particular, with a focus on routingissues in dynamic intersatellite link networks. He lectured on mobile satellitecommunications at Ilmenau Technical University, Ilmenau, Germany, from1995 to 1996, and on satellite system design in the M.Sc. Research SeminarSeries at University of Bradford, Bradford, U.K., in 2002. He is also apermanent Lecturer at the Carl-Cranz-Gesellschaft (CCG), Oberpfaffenhofen,Germany, teaching satellite communications courses for telecommunicationsprofessionals. He is coauthor of Satellite Systems for Personal and BroadbandCommunications (Berlin, Germany: Springer-Verlag, 2000).
