IEEE TRANSACTIONS ON BIG DATA 1

Taxi-Passenger-Demand Modeling Based onBig Data from a Roving Sensor Network

Desheng Zhang, Student Member, IEEE, Tian He, Senior Member, IEEE, Shan Lin, Member, IEEE,Sirajum Munir, Member, IEEE, and John A. Stankovic, Life Fellow, IEEE

Abstract—Investigating passenger demand is essential for the taxicab business. Existing solutions are typically based on offlinedata collected by manual investigations, which are often dated and inaccurate for real-time analysis. To address this issue, wepropose Dmodel, employing roving taxicabs as real-time mobile sensors to (i) infer passenger arriving moments by interactionsof vacant taxicabs, and then (ii) infer passenger demand by customized online training with both historical and real-time data.Dmodel utilizes a novel parameter called pickup pattern based on an entropy of pickup events (accounts for various real-worldlogical information, e.g., bad weather) to reduce the size of big historical taxicab data to be processed. We evaluate Dmodelwith a real-world 450 GB dataset of 14, 000 taxicabs for a half year, and results show that compared to the ground truth, Dmodelachieves 83% accuracy and outperforms a statistical model by 42%. We further present an application where Dmodel is used todispatch vacant taxicabs to achieve an equilibrium between passenger demand and taxicab supply across urban regions.

Index Terms—Taxicab System, Demand Modeling, Big Transportation Data.

F

1 INTRODUCTION

Understanding and predicting passenger demandare essential for the taxicab business [1]. With accu-rate knowledge of demand, taxicab companies canschedule their fleet and dispatch individual taxicabsto minimize idle driving time and maximize profits.Historically, such passenger demand has been inves-tigated by manual procedures (e.g., creating surveysor sampling [2]). However, these manual studies areoften dated, incomplete and difficult to use in realtime. In particular, though fairly stable duringlong-time periods (e.g., one day), passenger demandexperiences significant irregular spatio-temporal dy-namics during short-time periods (e.g., one hour)due to various real-world phenomena, e.g., badweather, accidents or special events. As a result,both long-term historical and short-term real-timedemand knowledge shall be utilized to capture suchdynamics. However, we face a challenge to create anaccurate demand model by combining both histori-cal and real-time demand, because historicaldemand is typically in a limited scale and real-timedemand is difficult to be obtained in real time [2].

• D. Zhang is with the Department of Computer Science and Engineer-ing, University of Minnesota, zhang@cs.umn.edu

• T. He is with the Department of Computer Science and Engineering,University of Minnesota, tianhe@cs.umn.edu

• S. Lin is with the Department of Electrical and Computer Engineering,Stony Brook University, shan.x.lin@stonybrook.edu

• S. Munir is with the Bosch Research and Technology Center, sira-jum.munir@us.bosch.com. This work was performed while the authorwas at the University of Virginia.

• J. A. Stankovic is with the Department of Computer Science, Univer-sity of Virginia, stankovic@cs.virginia.edu

In this work, we provide a two-part solutionbased on recent infrastructure updates of taxicabnetworks. First, we data mine a large dataset of his-torical information regarding passenger demand andtaxicabs’ trips. This results in the basis of ourmethod, from which we identify what aspectsshould be used to infer specific real-time demand. Inthis work, the historical GPS dataset used is from14,000 taxicabs for 6 months (450 GB) in a Chinesecity, Shenzhen. While this historical model is moreaccurate than surveys and sampling, it cannot han-dle many real-time issues and thus has majorlimitations if used alone.

Second, to address the short-term, real-time dy-namics, we consider thousands of roving taxicabs asreal-time mobile sensors and collect current informa-tion from them. This is possible because taxicabs indense urban areas are equipped with GPS as loca-tion sensors and fare meters as passenger sensors,and thus their locations and occupancy status can beperiodically uploaded to a dispatch center. Thesefrontend taxicabs and a backend dispatch centerform a real-time “roving sensor network”. Thestreaming data used are from a data feed in Shen-zhen taxicab network with an average rate of 450status records per second.

Admittedly, several systems have proposed to usetaxicab GPS traces to infer passengerdemand [3] [4] [5], but they typically have two sim-plifying assumptions: (i) they assume that previousdemand is given by picked-up passengers, but over-look waiting passengers who did not get picked up;and (ii) they assume that current demand can be in-ferred by long-term historical demand, but overlook

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 2

the fact that passenger demand is highly dynamic.For example, after a major concert, due to the highdemand, there are few picked up passengers yetnumerous waiting passengers, and the average his-torical demand cannot accurately indicate thesuddenly-increased demand due to the concert.

In this paper, to improve upon these two simplify-ing assumptions, we propose Dmodel, whichobserves hidden contexts to infer passenger demandbased on both historical and real-time taxicab data.The contributions of this paper are as follows.

• We identify passenger demand with a combinedoffline data analysis and a real-time roving sen-sor network, where taxicabs detect passengercounts and arriving moments. It is important tonote that passenger arriving moments are, ingeneral, unknown. But a major contribution ofDmodel is how the roving sensor network infersthem by utilizing taxicabs’ interactions.

• We present a novel parameter, called pickuppattern, to quantify taxicab operating similarityamong different daily data in a big taxicabdataset, e.g., 900 GB per year in Shenzhen. Notethat naively using more data from such a bigdataset results in not only unnecessary bigworkload but also inaccurate inferences. Thus,the key novelty of Dmodel is to utilize thereal-time pickup pattern to select customized yetcompact training data to increase inference accuracy.This pickup pattern implicitly accounts forspatio-temporal dynamics caused by real-worldphenomena, e.g., bad weather.

• We test Dmodel on a 450 GB dataset created by6 months of status records from 14, 000 taxicabsin Shenzhen. The evaluations show that com-pared to the ground truth, Dmodel achieves83% inference accuracy of demand in terms ofthe passenger counts, and outperforms a statisti-cal model by 42%. We will share such avaluable dataset for benefits of the big dataresearch community in the preprint version.

• We show Dmodel’s practical value in a realworld application where demand inferred byDmodel is used to dispatch vacant taxicabsacross city regions to achieve a better equilib-rium between passenger demand and taxicabsupply, which potentially leads to shorter idledriving times and higher profits for drivers aswell as shorter waiting times for passengers.

The rest of the paper is organized as follows. Sec-tion 2 gives our motivations. Section 3 shows aframework of modeling. Section 4 presents a rovingsensor network. Section 5 describes our model. Sec-tion 6 evaluates our model by a real-world 450 GBdataset. Section 7 presents the application based onour modeling, followed by the related work and theconclusion in Sections 8 and 9.

2 MOTIVATIONS

In this section, based on empirical data (introducedin Section 4) from a real-world taxicab network with14,000 taxicabs in Shenzhen, we present our motiva-tions to improve upon two legacy assumptions forpassenger demand analyses.

2.1 Assumption on Previous DemandLegacy Assumption One: Given a previous timeslot, the passenger demand (i.e., the total count of allpassengers requiring taxicab services) equals to thenumber of picked up passengers (i.e., pickupcounts) [3] [4] [5].

In this work, we argue that though all passengersget picked up eventually, for a previous slot the pas-senger demand should include not only picked-uppassengers but also waiting passengers who had ar-rived but did not get picked up. Figure 1 gives thedifference between pickup counts and total passen-ger counts, i.e., a pickup passenger count plus awaiting passenger count, for the entire Shenzhenarea in 5 minute slots (how to obtain these twocounts are given in Section 5.1).

0 2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 2 2 405 0 0

1 0 0 01 5 0 02 0 0 02 5 0 03 0 0 03 5 0 04 0 0 0

P i c k u p C o u n t T o t a l C o u n t

# of P

assen

gers

2 4 H o u r s

Fig 1: Pickup and Total Counts

We find that the pickup and total passenger countsare usually different, especially in the slots of therush hour. Thus, this assumption overlookingwaiting passengers leads to an inaccurate analysis.

The key reason for this assumption is that arrivingmoments for picked-up passengers cannot be ob-tained by existing infrastructures. To address thisissue, we present a novel method based on the inter-actions of vacant taxicabs to infer arriving moments,which are used to incorporate waiting passengersfor accurate passenger demand analyses. The detailsare given in Section 4.4.2.

2.2 Assumption on Current DemandLegacy Assumption Two: Given a current time slot,the passenger demand can be inferred by the previouspassenger demand for the same slot [5].

For this assumption, we argue that for the samearea and slot, the passenger demand experiences ir-regular temporal dynamics in different days due to

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 3

Day 1 Day 3 Day 2

A

B

A

BA

B

279

608

285

462

147

676

Fig 2: Demand Dynamics for the Same Hourly Slot in Three Different Days

various real-world factors, and cannot be accuratelyinferred without considering more contexts. Figure 2gives the passenger demand for the same hourly slotin three different weekdays, which is shown by totalpassenger counts in different administrative regionsof Shenzhen. Suppose we want infer the passengerdemand of Region A and B in Day 3 given in themiddle figure, and the historical demand for thesame regions and the same slot in Day 1 and Day 2is given by the left and right figures. If we infer Re-gion A’s demand in Day 3 based on the previousRegion A’s demand in Day 1, we only have279−|147−279|

279 ≈ 53% accuracy; similarly, if we inferRegion B’s demand in Day 3 based on the RegionB’s demand in Day 2, we only have608−|462−608|

608 ≈ 76% accuracy. Thus, the assumptiontwo leads to an inaccurate inference.

The key reason for this assumption is lacking aneffective parameter to select related historical data astraining data for the inference. Thus, in this paper, toimprove this assumption, we propose a novel pa-rameter called pickup pattern to select a customizedtraining dataset for a particular demand inference.For example, based on the pickup pattern, if we findthat Region A’s demand in Day 2 is more related toRegion A’s demand in Day 3, then we infer RegionA’s demand in Day 3 based on Region A’s demandin Day 2. As a result, we improve the accuracy forRegion A in Day 3 from 279−|147−279|

279 ≈ 53% to279−|285−279|

279 ≈ 98%. Thus, finding highly relateddata for the inference increases accuracy, and also re-duces the workload to process big taxicab data. Thedetails are given in Section 5.2.1.

2.3 Summary

The above two assumptions are the key reasons forinaccuracy of existing modeling methods for taxicabpassenger demand. In particular, an accurate infer-ence on previous demand is the key foundation foran accurate inference on current demand. To im-prove upon these two assumptions, we present amodeling method with a framework as follows.

3 FRAMEWORK

In this section, we present an overview of our mod-eling with three components, i.e., Roving SensorNetwork, Model Generation, Model Utilization as inFigure 3. These three components span the wholetaxicab-data-processing chain.

Historical &

Real-time

Data

Frontend Taxi System

Dispatching

System

Pickup

Event

Cruising

Event

Passenger Demand Model

Passenger

Arriving

MomentRoving

Sensor

Network

(Section 4)

Model

Generation

(Section 5&6)

Model

Utilization

(Section 7)

Backend Dispatch Center

Pickup

Pattern

Customized

Online

Training

Hidden

Markov

Chain

Fig 3: Framework of Dmodel

Roving Sensor Network. Recently, taxicab infras-tructures in large cities are upgraded with onboardGPS and communication devices as well as dispatchcenters [6]. From a broad perspective, we considerfrontend taxicabs and a backend dispatch center as aroving sensor network to infer passenger demand.We utilize both historical and real-time data fromsuch a network to detect two kinds of events, i.e.,pickup events and cruising events, to infer passen-ger arriving moments for our later model generationcomponent. The details are given in Section 4.

Model Generation. Based on the massive histori-cal and real-time data, we generate a passengerdemand model called Dmodel. Dmodel utilizes ahidden Markov chain to model passenger countsbased on real-time passenger arrival and pickups.During the modeling process, we propose a system

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 4

parameter called pickup pattern to obtain highly re-lated historical data for customized online trainingto indicate a relationship between pickup passengercounts and total passenger counts. Finally, Dmodeloutputs total passenger counts at road-segment lev-els during a fine-grained time interval. The details ofdesigns and evaluations for Dmodel are given inSection 5 and Section 6, respectively.

Model Utilization. According to passenger de-mand predicted by Dmodel, we propose areal-world application, i.e., a dispatching system toredistribute taxicabs among different urban regionsto achieve an equilibrium between passenger de-mand and transit supply at region levels, whichfinally provides feedback to the frontend taxicab sys-tem and closes the control loop. Such an applicationhas the potential to reduce passenger waiting timesand driver cruising miles, which is introduced andevaluated in Section 7.

With a highlight on passenger demand inferences,our modeling builds an architectural bridge betweengeneric taxicab infrastructures and real-worldknowledge output tailored by a specific application.

4 ROVING SENSOR NETWORKRecently, taxicab infrastructures in large cities havebeen updated with onboard GPS and communica-tion devices as well as a dispatch center to receiveGPS data from taxicabs in real time to monitorstatus of taxicab networks. Built on such an infras-tructure, a roving sensor network consists of (i)numerous roving taxicabs in the frontend as mobilesensors to detect passengers, and (ii) a dispatch centerin the backend to receive sensing data (i.e., taxicabstatus) from taxicab sensors to analyze demand. Inthis work, we utilize a taxicab network in Shenzhenwith 14,453 taxicabs and a ridership of more than200 million per year as an example to study such aroving sensor network. In a roving sensor network,taxicabs record their physical status, e.g., current lo-cation and speed, with GPS devices; taxicabs alsorecord their logical status, i.e., with passengers ornot, with fare meters; both their physical and logicalstatus is periodically (30 seconds on average)uploaded to dispatch centers with onboard commu-nication devices, in terms of sensing records. Asensing record mainly consists of the followingparameters: Plate Number; Date and Time; GPS Co-ordinates; Status Bit (1 or 0: indicating withpassengers or not when this record is uploaded).

Figure 4 gives a dataset about such sensing recordsfrom Shenzhen in China [7] (17, 150 people per squareKM). This half-year dataset contains almost 4 billionsensing records with a size more than 450 GB.

4.1 Data ManagementWe briefly introduce our management issues aboutbig data in the roving sensor network.

Collection Period 6 Months

Collection Date 01/01-06/30

# of Taxicabs 14,453

# of Pickup Events 98,472,628

# of Sensing Records 3.9 Billion

Data Size 450 GB

Sensing Dataset Summary

Fig 4: Dataset Summary

4.1.1 Data StorageWe establish a secure and reliable transmissionmechanism, which feeds our server taxicab GPS datacollected by Shenzhen Transport Committee by awired connection without impacting the originaldata source. Such a big amount of sensing data re-quires significant efforts for efficient storage andmanagement. In this project, we store the data byutilizing a 34 TB Hadoop Distributed File System(HDFS) on a cluster consisting of 11 nodes, each ofwhich is equipped with 32 cores and 32 GB RAM.For daily management, we typically use theMapReduce-based tools, e.g., Pig and Hive.

4.1.2 Data CleaningDue to the extremely-large size of our data, we findthree main kinds of errant data. (i) Missing Data:e.g., a taxicab’s sensing records were not uploadedwithin a given time period. Such missing data aredetected by monitoring temporal consistence of in-coming data for every taxicab. (ii) Duplicated Data:e.g., the sensing dataset shows two identical recordsfor the same taxicab. Such duplicated data are de-tected by comparing timestamps of every recordbelonging to the same taxicab. (iii) Data with LogicalErrors: e.g., GPS coordinates show that a taxicab isoff the road. Such data with logical errors are de-tected later when we analyze the data. In particular,we utilize a digital map of Shenzhen to verify if aGPS location is plausible or not. This is performedby checking the previous locations and durations be-tween timestamps of two records. The above errorsmay result from various reasons, e.g., hardware mal-functions, software issues, and communication. Toaddress the above errors, for all incoming data, wefirst filter out duplicated records and records withmissing or errant attributes. Then we correct obviousnumerical errors by various known contexts. Wenext store the data by dates and categories. Finallywe compare temporal consistence of the data to de-tect missing records. Admittedly, the missing orfiltered-out data (which accounted for 12% of the to-tal data) may impact the performance of our latermodeling, but we believe we are still able to provideinsightful analyses given the long time period of thedata we have collected.

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 5

4.2 Constrained Sensing Capability

Though our roving sensors produce anextremely-large sensing dataset, they haveseverely-constrained sensing capabilities. In regularsensor networks, the primary objective of sensors isto detect events, and thus sensors typically have afull sensing capability, i.e., sensors can be temporallyand spatially controlled to detect events. However, aroving sensor network consists of taxicabs whoseprimary objective is to deliver passengers, instead ofto detect passengers, resulting in a constrained sens-ing capability. Such capabilities are shown by asensing cycle of a taxicab in Figure 5 where a taxicabsensor functions in two alternating phases.

Phase I:Capable of

Detecting Passengers Picking up Passengers

from Vacant to Occupied

Detected a passenger

Dropping off Passengers

from Occupied to Vacant

Start to detect a passenger Phase II: Incapable of

Detecting Passengers

Fig 5: Roving Taxicab Sensing Cycle

• Phase I: Starting from dropping off a passenger(changing its status from occupied to vacant), ataxicab aims to find a new passenger and is ca-pable of detecting a passenger by roving onstreets until it picks up one.

• Phase II: Starting from picking up a passenger(changing its status from vacant to occupied), ataxicab aims to deliver this passenger and is in-capable of detecting new passengers until itdrops off this passenger.

According to the above two phases, we face threechallenges to design an inferring model based onroving sensor networks: (i) taxicab sensors can onlydetect passengers in their sensing cycles’ Phase Iwhere they cruise streets to look for passengers(Phase I accounts for roughly 40% of a taxicab dailyoperating time on average [8]), and after they pickup passengers and enter Phase II, they cannot detectother passengers waiting on streets (left-behind pas-sengers); (ii) taxicab sensors can only indicate thepassenger pickup moments, not arriving momentsthat have to be considered in demand modeling; (iii)taxicab sensors can only provide simple “on” or“off” status, which is difficult to be utilized to minereal world logical information that should be consid-ered in an inferring model, e.g., many waitingpassengers in extreme weather.

Even though with constrained sensing capabilities,roving taxicabs still provide substantial informationabout passengers to address these challenges. Suchinformation is given by both events detected by taxi-cabs and phenomena inferred from detected events,as shown by the following two subsections.

4.3 Detected EventsWe observe two kinds of events related to passengerdemand by tracking taxicabs’ sensing records.

4.3.1 Pickup EventIf a taxicab’s status turns from “unoccupied” to “oc-cupied” in two consecutive records, then it indicatesthat this taxicab just picks up a passenger in the loca-tion indicated by corresponding GPS coordinates,which is associated to a pickup event; similarly, adropoff event is indicated.

Figure 6 gives a daily pickup event distributionamong 495 Shenzhen city regions. A warmer colorindicates a higher number of pickup events.

Fig 6: Pickup Event Distribution in Urban Region

Figure 7 gives a graph representing the corre-sponding pickup and dropoff events in 245 majorurban regions in Shenzhen (including an airport,train stations, residential areas, etc) from 7AM to9AM of a Monday. The size of vertex indicates thenumber of events in the corresponding region; thecolor of vertex indicates one of six urban districts. Alink indicates passenger mobility patterns betweentwo regions, which are obtained by aggregating alltrips between these two regions. We remove thelinks with trips fewer than 30 for clarity.

Fig 7: Corresponding Pickup and Dropoff Events

4.3.2 Cruising EventA cruising event begins with a dropoff event and fi-nally ends with a pickup event. Figure 8 gives a

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 6

cruising event where a taxicab first drops off a pas-senger between l1 and l2, and then cruises from l2 tol3, and finally picks up a new passenger between l3and l4. By this cruising event, we infer an absence ofpassengers on the segment from l2 to l3 during thetime when this taxicab cruises it.

GPS Locations

Without Passengers

GPS Locations

With Passengers

Pickup

Dropoffl1

l2

l3

l4

Fig 8: Cruising Events

4.4 Inferred Phenomena

Based on the above two events, we study two inferredphenomena as follows.

4.4.1 Passengers on a Spatio-Temporal AreaPhenomenon 1 in Figure 9 gives a pickup event pi

where a vacant taxicab Ti cruised a segment sj andpicked up one passenger Pi. Based on this observa-tion, we infer that there is only one passenger (i.e.,Pi) in the dashed temporal and spatial area. This isbecause if there is another passenger Pj , Ti wouldpick Pj up, which contradicts to the fact that Ti

picked up Pi in the pickup event pi. Phenomenon 2in Figure 9 shows a cruising event where a vacanttaxicab Tj cruised a segment sj and did not pick upany passenger. Based on this observation, we inferthat there is no passenger in the dashed temporaland spatial area. This is because if there is a passen-ger Pj , Tj would pick Pj up, which contradicts thefact that Tj did not pick up passengers when itcruises sj . Note that there may be a passenger out-side the dashed area yet inside the rectangle, since apassenger can arrive at a location on segment sj , af-ter vacant taxicabs passed this location. This newlyarriving passenger cannot be detected until he/sheis picked up by another vacant taxicab.

4.4.2 Arriving Moments of Picked Up PassengersAn arriving moment indicates the time when a pas-senger starts to wait for a taxicab, which is used toobtain the ground truth of a total passenger countfor a segment during a slot. Accurately obtainingsuch arriving moments is almost impossible undercurrent infrastructures. But we present a method toobtain the upper bound of an arriving moment

Space sj

Time

li

Tj

tupper

pi

ti

Ti

t0

Space sj

Time

Tj

tjt0

Space sj

Time

Ti

tit0

pi

Phenomenon 1 Phenomenon 2 Phenomenon 3

Pj

Pj

Pk

Pi

Fig 9: Inferred Phenomena

tupper, i.e., the earliest possible moment of a passen-ger starting to wait for a taxicab. As in Phenomenon3 of Figure 9, supposing passengers do not movesignificantly when waiting for taxicabs, given apickup event pi in terms of pickup moment ti andlocation li, we find the latest cruising event whereanother vacant taxicab Tj passed the same location li(shown as the star). Thus, the moment tupper whenTj passed li is the upper bound of the arriving mo-ment of the passenger Pi in the pickup event pi.This is because if the moment that Pi starts to waitfor a taxicab is earlier than this bound tupper, then Pi

would be picked up by Tj at tupper, which contra-dicts the fact that Pi was picked up by Ti at ti. Weuse this upper bound as the arriving moment (e.g.,pushing more arriving passengers to earlier slots),which leads to a lower bound of the arrival countfor the latest slot, enabling a cautious inference.Note that waiting passengers’ arriving momentscannot be inferred until they are picked up.

Inferred by a roving sensor network, the abovephenomena provide abundant information with highresolutions, and are used by Dmodel to inferpassenger demand as follows.

5 MODEL GENERATION

Dmodel is a dynamic inference model for genericpassenger demand at road-segment levels on afine-grained temporal basis (e.g., one hour). Concep-tually, for a segment sj , at the end of a slot τi,Dmodel takes both real-time data uploaded in τi andhistorical data uploaded before τi as input, and pro-duces inferred demand in terms of a total passengercount for the next slot τi+1, by summing up twokinds of passengers as follows.

Previous Left-behind Passengers who had ar-rived at segment sj before the end of τi, and yetwere not picked up in τi. To obtain their count,Dmodel first aggregates real-time pickup events toobtain the pickup count for picked up passengers inτi. Next, Dmodel employs a novel parameter calledpickup pattern to obtain customized training data toinfer the total passenger count (either picked up ornot) in τi by corresponding pickup counts. Finally,Dmodel obtains the left-behind passenger count by

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 7

TABLE 1. Main NotationsNotation Descriptionpi Pickup Event iPi Passenger iTi Taxi iti Pickup Time in Pickup Event ili Pickup Location in Pickup Event iai Passenger Arriving Event for Pickup Event iτi Modeling Time Slot idx Day xsj Road Segment j

Psjτi

Pickup Count during τi at sj

Lsjτi

Left-behind Count during τi at sj

Asjτi

Arrival Count during τi at sj

Tsjτi

Total Passenger Count during τi at sj

ρsjτi

Pickup Pattern τi at sj

subtracting the pickup count of τi from the totalpassenger count of τi.

Future Arriving Passengers who have not arrivedyet but will arrive during τi+1 at segment sj , i.e., thefuture arrival. Dmodel infers the future arrival bymaintaining a probability distribution of a passengerarrival rate for every road segment. At the end of aslot τi, based on the pickup count in τi, Dmodel firstinfers the corresponding passenger arrival in τi, andthen updates the distribution of arrival rates accord-ingly, and finally infers the future arrival by thisupdated distribution.

As follows, we present passenger demand model-ing, and then elaborate how to obtain these twokinds of passengers.

5.1 Passenger Demand ModelingThe notations used in this paper are given in Table I.Four key notations for a slot τi and a segment sj aregiven as follows.• Psj

τi : Pickup Count: The total number of pickedup passengers during τi at sj .

• Lsjτi : Left-behind Count: The total number of

waiting yet not picked up passengers during τi

at sj .• Asj

τi : Arrival Count: The total number of arrivingpassengers during τi at sj .

• Tsjτi : Total Passenger Count: The total number of

passengers who wait for taxicabs during τi at sj .As follows, we omit all same superscripts for a concisenotation. Figure 10 shows examples of the notations.The x-axis is the time, and the y-axis is the space,i.e., segment sj . A total of three passengers is pickedup, indicated by three pickup events p1, p2 and p3.Further, three arriving events that they start to waitfor taxicabs are given by a1, a2 and a3. As a result,for the time slot τ0, Aτ0 = 1, Pτ0 = 0, Lτ0 = 1, Tτ0 = 1;for the time slot τ1, Aτ1 = 2, Pτ1 = 2, Lτ1 = 1, Tτ1 = 3;for the time slot τ2, Aτ2 = 0, Pτ2 = 1, Lτ2 = 0, Tτ2 = 1.Given pickup points, arriving points, and waitingperiods as shown by dots, stars, and dashed lines be-tween stars and dots in Figure 10, Pτi

, Aτi, or Tτi

fora time slot τi are obtained by as simply as counting

dots, stars, or dashed lines, respectively. Note thatalthough some passengers are double-counted at dif-ferent slots, all passengers are counted once at thesame slot.

Space sj

Time

p1

Slot τ1

p3p2

t2Slot τ2

t1

a1

a2 a3

t3

Pickup Event

Arriving Event

Waiting Period

Trace with Passengers

Trace without Passengers

Slot τ0

t0

Fig 10: Notation Example

5.1.1 Demand Modeling by a Hidden Markov ChainIn Figure 11, we analyze passenger demand as anunobservable state in a Hidden Markov Chain.

Total Passenger Count Tτi Tτi+1 Tτx

Aτi Aτi+1

Pτi Pτi+1Pickup Count

Arrival Count

Lτi-1 Lτi Lτi+1Left-Behind Count

1)

3)

2)

Fig 11: Passenger Demand in a Hidden Markov Chain

1) At the end of a slot τi, the key system state thatneeds to be inferred is the total passenger countTτi+1 of the next slot τi+1, which takes the left-behind count Lτi

of τi and the arrival count Aτi+1

of τi+1 as two inputs (shown by the arrows withsolid lines). Thus we have

Tτi+1 = Lτi+ Aτi+1 .

2) As one input for Tτi+1 , the left-behind countLτi of τi is also one of two outputs (shown bythe arrows with dashed lines) of the previoussystem state, i.e., the total passenger count Tτi

of τi. The other output of Tτiis the observable

pickup count Pτiof τi. Thus we have

Lτi= Tτi

− Pτi.

3) As the other input for Tτi+1 , the arrival countAτi+1 of τi+1 is inferred by a stochastic process,supposing passengers arrive according to ageneric Poisson process.

4) Thus, combining two equations together, wehave our key inferring equation as follows.

Tτi+1 = (Tτi− Pτi

) + Aτi+1 . (1)

5.1.2 Inference OverviewAs in Figure 12, at the end of every slot, e.g., currenttime ti+1, Dmodel infers Tτi+1 for a segment sj byEq.(1) with four steps as follows.

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 8

Time

Slot τiti+1 Slot τi+1

Pτi

Tτi

Aτi+1

Real Time &

Historical

Sensing

Dataset

Current Time

Previous Slot Future Slot

1)

2)

3)

4)

Tτi+1

Fig 12: Inference Overview

1) It infers pickup count Pτi by aggregating pickupevents in the latest slot τi from real-time data;

2) It infers total passenger count Tτibased on the

corresponding pickup count Pτiand a cus-

tomized corrective model trained by bothhistorical and real-time data.

3) It infers arrival count Aτi+1 for the next time slotτi+1 by a probability distribution D of passengerarrival rate λ at segment sj , which is periodicallymaintained through a Bayesian updating basedon pickup count Pτi

.4) It infers total passenger count Tτi+1 for the next

slot τi+1 with Eq.(1), by Pτi , Tτi and Aτi+1 .In the above steps, steps 1) and 4) are straightforward,so we elaborate steps 2) and 3) in Section 5.2 and 5.3.

5.2 Inferring Total Passenger Count Tτi

We first introduce our key novelty about using pickuppatterns, and then propose how to infer Tτi .

5.2.1 Pickup PatternIn this work, we infer the total passenger count byfour factors, which include (i) time in terms of a timeslot of a day (e.g., slot τi), (ii) location in terms of aroad segment (e.g., segment sj), (iii) pickup count interms of how many passengers have been picked ona segment during a slot, and (iv) pickup pattern interms of how fast passengers were picked up, whichmay infer hidden contexts, e.g., extreme weather ormajor events. The existing work has been consider-ing the first three factors, but the pickup pattern hasnot been considered by others before. In this work,we argue that the pickup count is inherently limitedby taxicab supply, and cannot provide enough infer-ring information. But our pickup patterns provideextra hidden contexts to increase inference accuracy.

Figure 13 presents the same slot τi at two differentdays with the same pickup count yet with differentpickup patterns. The key difference of the same slotτi for day dx and dy is how long it takes for vacanttaxicabs to pick up passengers during τi, which isassociated to the pickup pattern, i.e., the taxicabs in dx

pick up two passengers very quickly; whereas thetaxicabs in dy cruise for a long time before picking

ti+1t1

Space sj

ti+1ti

p1

Day dx Day dy

p2

Slot τi Slot τi

Space sj p1

p2

Fig 13: Pickup Patterns

up two passengers. The pickup pattern gives us ex-tra hidden online contexts, and cannot be replacedby other contexts already used by the existing work,i.e., slot τi, segment sj , and pickup count Pτi∈dz

of aparticular day dz , since in Figure 13, all other con-texts are the same, but two slots τi in dx and dy havedifferent pickup patterns. For example, the hiddenonline contexts in the pickup pattern during τi ∈ dx

may indicate suddenly increased demand due to ex-treme weather, train arrival or other events, since alltaxicabs pick up passengers very quickly. Whereasthe pickup pattern for τi ∈ dy may indicate a normalscenario without increased demand. Intuitively,though dx and dy have same pickup countPτi∈dx

= Pτi∈dy= 2 (may result from limited taxicab

supply), τi ∈ dx shall have a larger total count Tτi∈dx

than τi ∈ dy .To quantify the pickup pattern as a formal param-

eter, as in Figure 13, we first use a random variablerpi

to indicate the area ratio between the dashed tri-angle spatio-temporal area introduced by a pickupevent pi and the entire rectangle spatio-temporalarea. For example in Figure 13, suppose the dashedspatio-temporal triangle area | 4p1 | associated withp1 during slot τi on Day dx is equal to 1, and the en-tire spatio-temporal rectangle area |sj | × |τi| is equalto 16. Thus, rp1 is equal to |4p1 |

|sj |×|τi| = 116 . By intro-

ducing this ratio, we integrate both the pickuplocation and time for a pickup event p1 by a randomvariable, because different pickup locations andtimes in the same spatio-temporal context lead todifferent areas of dashed triangles, and thus lead todifferent area ratios. Further, we integrate all pickupevents {p1, p2, ..., pn} associated with a givenspatio-temporal context (i.e., during a given slot τi

on a given road segment) by an integration ofassociated random variables {rp1 , rp2 , ...rpn

}. In par-ticular, our integration is based on the entropy ρ ofthese random variables to indicate the pickuppattern under this spatio-temporal combination.

ρτi = −n∑

i=1

rpi log rpi

where n is the total number of pickup events during

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 9

slot τi in this road segment. A low entropy shows alow randomness of area ratios associated with thesepickups, which indicates pickups always happenedat similar locations of a given segment sj duringsimilar times of a slot τi, and thus leads to similararea ratios. As in evaluations, ρ accounts for manyreal-world scenarios that cannot be captured withpickup counts due to limited taxi supply.

5.2.2 Customized Online TrainingBased on the new online factor, pickup pattern, andother three factors, we discuss how to infer the totalpassenger count as follows.

Given a segment sj and a slot τi, the pickup countPτi

and the total passenger count Tτihave a logical

relationship: Pτiis the lower bound of Tτi

, since allpicked up passengers are included in the total pas-senger count. Thus, we quantitatively investigatetheir relationship as follows. Given the historicaldataset, for particular slots and segments, we obtainthe ground truth of Pτi

by aggregated pickup events,and infer the ground truth of Tτi

based on themethod of inferring arriving moments (introduced inSection 4.4.2), e.g., in Figure 10, after inferring arriv-ing moments (shown by stars), Tτi

are obtained bycounting dashed lines linking dots and stars in a slot(e.g., Tτ1 = 3). As a result, Figure 14 gives the rela-tionship between P and T for 10 randomly selectedroad segments in five τ8 slots from 8 to 9AM in fiveweekdays. It indicates an approximate linearrelationship for Pτ8 and Tτ8 for the same segment sj .

2 5 5 0 7 5 1 0 0 1 2 5 1 5 0 1 7 5 2 0 0 2 2 52 55 07 5

1 0 01 2 51 5 01 7 52 0 02 2 5

R S 1 R S 2 R S 3 R S 4 R S 5 R S 6

R S 7 R S 8 R S 9 R S 1 0

Total

Passe

nger

Coun

t (T)

P i c k u p C o u n t ( P )

Fig 14: T vs. P in 10 Segments

Based on the above observation, we propose acustomized online training model based on the lin-ear regression as follows. Supposing that (i) we havea historical dataset consisting of taxicab GPS dataabout K − 1 different days, i.e., day d1 to day dK−1,and (ii) the current time is the end of slot τi in daydK , Dmodel infers the total passenger count Tτi∈dK

with four steps as follows.1) It calculates both pickup count Pτi∈dK

and thecorresponding pickup pattern ρτi∈dK

, based onreal-time data about the latest slot τi ∈ dK .

2) It selects the data of days whose τi have similarpickup pattern ρ to ρτi∈dK

as a customizedtraining dataset with M pairs of

(Pτi∈dm, Tτi∈dm

) where 1 ≤ m ≤ M (one pairfor every day).

3) It trains the following model by the M pairs of(Pτi∈dm

, Tτi∈dm) to learn customized ατi∈dK

andβτi∈dK

.

Tτi∈dm = ατi∈dK+ βτi∈dK

× Pτi∈dm . (2)

4) It utilizes ατi∈dK, βτi∈dK

and pickup countPτi∈dK

to obtain total passenger count Tτi∈dK

with Eq.(2).A similar pickup pattern ρ to ρτi∈dK

is defined byρ ∈ [ρτi∈dK

· (1−∆ρ), ρτi∈dK· (1 + ∆ρ)] where ∆ρ is a

given parameter and carefully evaluated in Section 6.

5.3 Inferring Arrival Count Aτi+1

Dmodel infers passenger arrival with a stochasticprocess where an arrival rate λ of a Poisson Processvaries in Brownian motion, which is widely used tomodel passenger arrival or network package ar-rival [9]. Thus, Aτi+1 = λτi+1 × |τi+1|. Note that wedid not use a customized training to infer the arrivalcount Aτi+1 based on given pickup count Pτi as inthe last subsection, since there is no potentiallylogical relationship between Pτi

and Aτi+1 .

5.3.1 Passenger Arrival Rate ModelingDmodel maintains a probability distribution D of λfor a segment sj by discretizing the space of passibleλ, and assumes that (i) λ is one of discrete valuesfrom 0 to the maximum λ (obtained by the dataset)and (ii) the initial probability for all possible λ isuniformly distributed. Therefore, at the end of theslot τi, Dmodel updates D with three steps.

1) It evolves D to the current time by applyingBrownian motion to every possible rate by as-suming that λ is undergoing a continuous-timestochastic process.

2) It infers the arrival count Aτiin τi based on ob-

served Pτi, and calculates probabilities that this

arrival count Aτiis associated to every one of

arrival rates as follows.

F (x)← Dold(λτi= x)× e−x·|τi| (x · |τi|)Aτi

Aτi!

.

3) It normalizes these probabilities, so they sum tounity.

Dnew(λτi= x)← F (x)∑

k F (k).

These three steps constitute Bayesian updating for D.Given D, we try to infer λτi+1 with a cautious estimateto bound a risk of overinferring. So, we employ theωth percentile of D to calculate the inferred λτi+1 , e.g.,40th percentile. In Dmodel, ω is a given parameter,and is evaluated in Section 6.

A key unresolved question is how to infer the ar-rival count Aτi

by the pickup count Pτi, which is

introduced as follows.

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 10

5.3.2 Inferring Previous Arrival Count Aτi

We introduce how to infer Aτiin Figure 15 where

we classify all passengers associated to the total pas-senger count Tτi

of a slot τi into four parts, based onwhen they arrived and whether they got picked upat the end of slot τi. Thus, the sum of passengers inPart 1 and Part 2 is the arrival count Aτi we try toinfer. The sum of passengers in Part 2 and Part 3 isthe pickup count Pτi

; the sum of passengers in allfour parts is the total passenger count Tτi

; we havealready obtained both of them in the previoussubsection.

Part 2:

Arrived in τi

Picked up in τi

Part 3:

Arrived before τi

Picked up in τi

Part 1:

Arrived in τi

Left-Behind after τi

Part 4:

Arrived before τi

Left-Behind after τi

Total Passenger Count of Slot τi

Pickup Count of Slot τi

Arrival Count of Slot τi

(P )τi

(T )τi

(A )τi

Fig 15: Inferring Previous Arrival Count Aτi

We add the following two kinds of passengers to inferAτi

.1) Passengers in Part 1: Since we have already in-

ferred the total passenger count Tτibased on

Pτiin Section 5.2, we have the total number of

passengers in Parts 1 and 4 together, i.e.,Tτi − Pτi . Further, since an inferring slot (e.g.,one hour) is typically longer than a passengerwaiting period, the number of passengers inPart 4 is 0. Thus we have the number ofpassengers in Part 1 alone.

2) Passengers in Part 2: We differentiate the pas-sengers in Parts 2 and 3 by inferring arrivingmoments of these picked up passengers in Pτi

,based the method in Section 4.4.2, and we ob-tain the number of passengers in Part 2alone.

6 DMODEL EVALUATION

We evaluate Dmodel based on a 450 GB dataset in-troduced in Section 4. We divide the entire 182-daydataset into two subsets. Testing Dataset: it containsthe data about a particular day, e.g., day d1, servingas the real-time streaming data in the evaluation.Training Dataset: it contains the data about the rest ofdays, serving as the historical training data. For thisparticular day d1, if we use one-hour slots, at theend of the first slot, i.e., time 01:00, we use Dmodelto infer the total passenger count for the next slotfrom 01:00 to 02:00, based on both the “real-time”data from 00:00 to 01:00 in the testing dataset, andall data in the historical training data. We let thetesting dataset rotate among all 182 days of data,leading to 182 sets of experiments. The averageresults were reported.

We compare Dmodel with two models: SDD andBasic. SDD model is one of the state-of-the-art taxi-cab demand and supply models, which maintains adistribution for passenger demand based on the pre-vious average demand [4]. SDD model serves as astatistical model and is suitable for the real worldscenario where the real-time data collection is notpossible, and we can only use the historical data toinfer passenger demand. Basic model first uses thegeneric offline training to train the entire dataset toobtain parameters (α and β) offline without consid-ering real-time pickup patterns. Basic serves as abaseline for Dmodel to show the effects of the igno-rance of logical contexts shown by pickup patterns(e.g., extreme weather or events) on the model per-formance. Dmodel performs similarly with Basicexcept that it uses logical contexts (pickup and cruis-ing events) in the testing dataset to calculate apickup pattern for a particular slot, and selects thedata of slots with similar pickup patterns as a cus-tomized training dataset to perform an onlinetraining as introduced in Section 5.2.2.

By processing the entire dataset with a method ofinferring passenger arrival moments (introduced inSection 4.4.2), we infer the ground truth of total pas-senger counts used to test models with a key metric,called Accuracy. The accuracy is defined as a ratio= T−|T−T|

T where T is the inferred total passengercount of a particular model and T is the total pas-senger count obtained from the inferred groundtruth.

We first test models on 4 and 1000 road segmentsabout accuracy with different slot lengths. Then, weinvestigate the sensitivity of Dmodel to two key pa-rameters: ∆ρ and ω used in Sections 5.2.2 and 5.3.1,and obtain their optimal default values. Next, westudy impacts of lengths of historical data on modelaccuracy and model running times. Finally, wepresent a summary.

6.1 Inference AccuracyIn this subsection, we show low-level comparisonson 4 particular road segments, and high-level com-parisons on 1000 road segments. All road segmentsare randomly selected in the downtown area ofShenzhen.

6.1.1 Low-Level ComparisonsFigure 16 plots accuracy of three models on 4 roadsegments under one-hour slots. Dmodel has a betterperformance than Basic and SDD, especially at thenon-rush hour, e.g., 18:00 to 06:00. Basic outperformsSDD in the early morning, e.g., 00:00 to 06:00, andthe late night, e.g., 18:00 to 00:00. SDD has good ac-curacy during the morning rush hour, e.g., 08:00 to12:00, and we believe this is because during the rushhour, passenger demand is relative stable compared

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 11

6 1 2 1 8 0 6 1 2 1 8 0 6 1 2 1 8 0 6 1 2 1 8 04 0

6 0

8 0

1 0 0

0

S D D B a s i c D m o d e lAc

curac

y (%

)

2 4 H o u r s o f a D a y

R o a d S e g m e n t 1 R o a d S e g m e n t 2 R o a d S e g m e n t 4R o a d S e g m e n t 3

Fig 16: Accuracy under One Hour Slot for 24 Hours in Four Road Segments

to other time periods. We notice that performancesof different models are also dependent on locations,e.g., in road segment 1, Basic outperforms SDD dur-ing almost all the morning, but in road segment 3,SDD has a better performance roughly from 10:00 to23:00. Further, we also observe that even thoughDmodel has a better performance in general, butduring some hours and in some locations, Basic in-deed has a better performance, such as at roadsegment 3 from 18:00 to 19:00. This may be becausefor this road segment and time period, Dmodel didnot obtain a good parameter from the training dataselected according to pickup patterns.

6 1 2 1 8 0 6 1 2 1 8 0

6 0

8 0

1 0 0

0

S D D B a s i c D m o d e l

Accu

racy (

%)

2 4 H o u r s o f a D a y

R o a d S e g m e n t 1 R o a d S e g m e n t 2

Fig 17: Accuracy under Two Hour Slot for 24 Hours

Figure 17 shows comparisons in segments 1 and 2under two-hour slots. With a longer slot, the accu-racy generally increases for all three models. This isbecause (i) passenger demand is more stable in alonger slot, and thus SDD model becomes moreeffective; (ii) a longer slot increases accuracy of pas-senger arrival predictions in Dmodel and Basic,which leads to increased inference accuracy. We alsonotice that in a longer slot, the performance gain be-tween Dmodel and others increases, and this may bebecause the advanced online training used byDmodel is more effective for a longer slot.

6.1.2 High Level ComparisonsFigure 18 gives the average accuracy on 1000 roadsegments under one-hour slots at different hours ofa day. The average accuracy of all three models on1000 road segments is lower than the accuracy weobserved on 4 particular road segments. It is becausepassenger demand may change dramaticallybetween different segments. But the relative perfor-mance between three models is similar to Figure 16.

Basic outperforms SDD at the most of the time by18% on average, except at the evening rush hourwhere SDD outperforms Basic by 5%. Dmodel has abetter performance than SDD and Basic by 42% and13% on average, which results from its customizedonline training. In addition, we find that SDD modelhas a poor performance during the non-rush hourwhen passenger demand is not stable. But Dmodelovercomes this issue by its effective inferring.Dmodel has 83% accuracy at the 9AM slot, which isthe default slot for the following experiments.

2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 25 0

6 0

7 0

8 0 S D D B a s i c D m o d e l

Accu

racy (

%)

2 4 H o u r s o f a D a y

Fig 18: Average Accuracy 1 2 3 4 5 6 7 8

3 04 05 06 07 08 09 0

1 0 0

0 . 2 5 0 . 5 1 2 3 4 6 1 2

S D D B a s i c D m o d e lAc

curac

y (%

)L e n g t h o f S l o t ( H o u r s )

Fig 19: Effects of Slot Lengths

6.2 Sensitivity of DmodelWe study the sensitivity of Dmodel to slot lengths andtwo parameters ∆ρ and ω on 1000 segments.

6.2.1 Slot Length vs. AccuracyFigure 19 gives the average accuracy on 1000segments with different slot lengths. The average ac-curacy of all models increases with the lengths ofslots. The increasing on accuracy slows down whenslots are longer than 2 hours. This is because passen-ger demand in a longer slot becomes more stable atdifferent days. But when a slot is short, e.g., 15 minsor 30 mins, the average passenger demand is vari-able at different days. When the slot becomes longer,Dmodel and Basic have the similar performance, be-cause pickup patterns for long slots are mostlysimilar, and cannot be used by Dmodel todifferentiate related slots.

6.2.2 ∆ρ vs. Accuracy∆ρ is used to decide similarity between pickup pat-terns as in Section 5.2.2. Figure 20 gives effects of ∆ρon Dmodel. With the increase of ∆ρ, the accuracy of

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 12

Dmodel increases first, and then decreases. This isbecause when ∆ρ increases, Dmodel finds more slotswith similar pickup patterns to effectively train acustomized corrective model online. But when ∆ρbecomes too large, Dmodel has to consider moreslots with different pickup patterns, leading to apoor performance. Further, when ∆ρ becomes largerthan 0.5, Dmodel has to consider many slots, similarto Basic model. The accuracy peaks when ∆ρ = 0.2,which is set as the default value of ∆ρ. If the used∆ρ leads to an empty training dataset, ∆ρ increasesuntil the training dataset is not empty.

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 05 56 06 57 07 58 08 59 0

Accu

racy (

%)

M e a s u r i n g R a t i o D r ( % )

Fig 20: ∆ρ vs. Accuracy

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 03 04 05 06 07 08 09 0

Accu

racy (

%)

P e r c e n t i l e w ( % )

Fig 21: ω vs. Accuracy

6.2.3 ω vs. Accuracyω is used to decide the percentile to predict the fu-ture passenger arrival as in Section 5.3.1. Figure 21plots effects of ω on Dmodel. A small ω indicatesthat Dmodel conservatively predicts arrival rates;whereas a large ω indicates Dmodel aggressivelypredicts arrival rates. We find that both a small andlarge ω lead to a poor performance, since a small orlarge predicted passenger arrival rate reduces the ac-curacy of Dmodel. The accuracy peaks whenω = 0.4, which is set as the default value of ω.

6.2.4 Impact of Impact of Historical DataIn this subsection, we study the impact of historicaldata h in terms of weeks on model accuracy andrunning times by comparing Dmodel to Basic with adefault value of 26 weeks. Normally, the more thehistorical data, the more accurate the models. How-ever, naively using more data actually reducesmodel accuracy. Figure 22 plots the accuracy of Basicand Dmodel with different lengths of historical datain terms of weeks. We find that Dmodel always hasa better performance than Basic after the length ofhistorical data is longer than one week. This is be-cause the pickup pattern Dmodel used selects highlyrelated data to infer demand, which has higheraccuracy than Basic using all data for training.

Figure 23 plots running times of Basic andDmodel with different lengths of historical data interms of weeks. We find that Dmodel always hasshorter running times after the length of historicaldata is longer than 3 weeks. This is because calcula-tions for pickup patterns (e.g., calculating entropy) inDmodel take a higher portion of running timeswhen the length of historical data is short. But whenthe length of historical data becomes longer, these

0 5 1 0 1 5 2 0 2 53 04 05 06 07 08 09 0

B a s i c D m o d e l

Accu

racy (

%)

S i z e o f H i s t o r i c a l D a t a h ( w e e k s )

Fig 22: h vs. Accuracy

0 3 6 9 1 2 1 5 1 8 2 1 2 4 2 703 06 09 0

1 2 01 5 01 8 02 1 02 4 0

B a s i c D m o d e l

Runn

ing Ti

mes (s

)

S i z e o f H i s t o r i c a l D a t a h ( w e e k s )

Fig 23: h vs. Running Time

calculations only account for a small portion of run-ning times. However, Basic requires longer runningtimes because it uses all historical data for training.

6.3 Dmodel Evaluation SummaryWe made the following observations based on re-sults. (i) The accuracy of inferring models is highlydependent on both locations and times as in Fig-ures 16 and 17. On average, different models havedifferent accuracy at different times as in Figure 18.Compared to locations, all models are more sensitiveto time. (ii) System parameters pose significant im-pacts of accuracy, and the optimal parameters haveto be carefully evaluated. The length of slots has sig-nificant impacts on the relative performance betweenall models as in Figure 19. It is intuitive that alonger slot has better accuracy for all models, but alonger slot also has a low inferring usability formany applications. As in Figure 19, it seems that thetwo-hour slot is a good tradeoff between the usabil-ity and the accuracy of the real-time inferring model.Further, as in Figures 20 and 21, both ∆ρ and ω haveimpacts on accuracy of models, and ω has a biggerimpact compared to ∆ρ. (iii) By selecting a compactsize of highly related data, we increase model accu-racy and reduce running times at the same time asshown by Figures 22 and 23. (iv) Compared to mod-els statistically inferring demand with only historicalaverage demand data, models using real-timepickup events have better inferring accuracy, whichis shown by the fact that Dmodel and Basic outper-form SDD as in Figures 16, 17 and 18. (v) Takinglogical information (weather or events) shown bypickup patterns into considerations further increasesmodel accuracy, which is shown by the fact thatDmodel outperforms Basic as in the most figures.

7 DMODEL APPLICATION

We propose a Dmodel application where a dispatchcenter employs demand inferred by Dmodel toachieve an equilibrium between passenger demandand taxicab supply, given 245 major urban regions inShenzhen as shown in Figure 7.

In our application, at the end of a “real-time” slotτi, we first use Dmodel to infer passenger demandTrx

τi+1for the next slot τi+1 in region rx by aggregat-

ing inferred demand of all road segments in region

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 13

rx. Next, we employ real-time data to aggregate va-cant taxicabs in region rx to obtain “dispatchable”vacant taxicab supply in region rx for the next slotτi+1, indicated by Srx

τi+1. Similarly, we shall have all

Trxτi+1

and Srxτi+1

where 1 ≤ x ≤ 245. Finally, since ourpaper is focused on modeling, we use a straightfor-ward scheme to dispatch vacant taxicab supply∑

1≤x≤245 Srxτi+1

among 245 regions so that the dis-patched taxicab supply Srx

τi+1is proportional to

inferred passenger demand Trxτi+1

in region rx.The evaluation is based on the ground truth of

passenger demand Trxτi+1

of slot τi+1, and the dis-patched vacant taxicab supply Srx

τi+1in each region at

hourly slots. We propose a normalized equilibriumvalue 0 ≤ κ ≤ 1 to evaluate effectiveness of dispatch-

ing: κτi+1 = avg1≤x≤245

|Trxτi+1

−Srxτi+1

|

Trxτi+1+Srx

τi+1. If the demand

inferred by an inference method, e.g., Dmodel, issimilar to the ground truth, the correspondingdispatch leads to a small κτi+1 , indicating an equilib-rium between passenger demand and taxicabsupply; otherwise, it leads to a large κτi+1 , i.e., dise-quilibrium. Note that dispatching taxicabs wouldskew historical taxicabs’ GPS dataset. To eliminatedispatching effects, we only used dispatched supplyto calculate κ, and did not manipulate taxis’ traces,and we start over at the end of the next slot.

2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 21 0

2 0

3 0

4 0

5 0 S D D - B a s e d B a s i c - B a s e d D m o d e l - B a s e d

Equil

ibrium

Value

k (%

)

2 4 H o u r s o f a D a y

Fig 24: κ in One-hour Slots

2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 2 2 41 0

2 0

3 0

4 0

5 0 S D D - B a s e d B a s i c - B a s e d D m o d e l - B a s e d

Equil

ibrium

Value

k (%

)

2 4 H o u r s o f a D a y

Fig 25: κ in Two-hour Slots

Figure 24 plots the equilibrium value κ at differenthours of a day under one-hour slots. We observethat the equilibrium values fluctuate in all dispatch-ing. But Dmodel-based dispatching has a lowerequilibrium value almost at every slot. Basic-baseddispatching outperforms SDD-based dispatching atthe most of the time. Figure 25 gives the averageequilibrium value κ under two-hour slots. We findthe equilibrium values under two-hour slots arelower than the equilibrium values under one-hourslots for all dispatching, which verifies our previousobservation that two-hour-slot based inferring isbetter than one-hour-slot based one in terms of accu-racy. But the relative performance between threemodel-based dispatching shown in Figure 25 is simi-lar to the one in Figure 24. Basic outperforms SDDat the most of the time, except at the morning rushhour where SDD outperforms Basic by 5% on aver-age. Dmodel-based dispatching outperformsBasic-based and SDD-based dispatching by 11% onaverage, because of accurate inferring by Dmodel.

8 RELATED WORK

The method to infer passenger demand with historyis not new, but it is normally performed by usingsurvey [2] [8] and static sensor data [10] [1]. In thesemodels, both times and locations for inference arepreset, and the data used for inference is often in-complete and out-of-date. Recently, several novelsystems have been proposed using taxicab traces.

Some systems are proposed to assist taxicab opera-tors for better taxicab services, e.g., inferring mobilitypatterns for taxicab passengers [11], exploring car-pooling opportunities [12], dispatching taxicabsbased on inferred passenger demand [3] [4] [5], de-tecting anomalous taxicab trips to discover driverfraud [13], and discovering temporal and spatialcausal interactions to provide timely and efficientservices in certain areas with disequilibrium [14].

In addition to taxicab operators, several systemsare proposed for the benefit of passengers or drivers,e.g., allowing taxicab passengers to query the ex-pected duration and fare of a planed trip based onprevious trips [15], computing faster routes by tak-ing into account driving patterns of taxicabsobtained from historical GPS trajectories [16], esti-mating city traffic volumes for drivers [17], andrecommending a taxicab driver with a sequence ofpick-up points to maximize profits [3].

Moveover, taxicab GPS records help beyond thetaxicab business: (i) GPS records from experiencedtaxicab drivers can assist other drivers improve theirdriving performance [6]; (ii) GPS records can beused for navigating regular drivers to smart routesbased on those of experienced taxicab drivers [18];(iii) large-scale taxicab GPS traces enable us to betterunderstand traffic conditions of cities [19] [20].

Yet the most of existing research on taxicab sys-tems is mainly focused on taxicab scheduling,instead of passenger modeling, assuming that pas-senger demand is given by historical average pickupevents, and overlooking the fact that real-time de-mand is different from pickup events for the sametime period [21]. As a result, our model is differentfrom the existing research by its novel inferencemethod based on both real-time and historical datafrom roving sensor networks. Technically, we focuson inferring passenger demand with the compact yetcustomized online training with real-time pickuppatterns and hidden contexts (e.g., arriving mo-ments) inferred by roving taxicab sensors, whichhave not been investigated before.

9 CONCLUSION

In this work, we motivate, design and evaluate ataxicab passenger model Dmodel and one of its ap-plications based on a 450 GB dataset collected by ataxi system as a roving sensor network. Our effort

D.ZHANG et al.: TAXI-PASSENGER-DEMAND MODELING BASED ON BIG DATA FROM A ROVING SENSOR NETWORK 14

provides a few valuable insights for applying model-ing techniques in Dmodel to other transportationsystems. Specifically, (i) mobile taxicabs can be usedas roving sensors to infer passenger demand withhigh accuracy; (ii) the inferring accuracy is highlydependent on locations, times, and other logical in-formation, e.g., weather and events; (iii) the lengthof inferring slots also has significant impacts on theinferring accuracy, and a good tradeoff between theusability and accuracy of demand methods has to becarefully evaluated; (iv) a statistic model can be en-hanced by a generic offline training consideringpickup events, but it can be further enhanced by acustomized online training for real-time situations.

REFERENCES[1] “National transport authority,” in National Taxi Fare Review

2012.[2] “San francisco municipal transportation agency:taxi user sur-

veys,” in San Francisco Municipal Transportation Agency.[3] Y. Ge, H. Xiong, A. Tuzhilin, K. Xiao, M. Gruteser, and M. Paz-

zani, “An energy-efficient mobile recommender system,” inKDD ’10.

[4] Y. Huang and J. W. Powell, “Detecting regions of disequilib-rium in taxi services under uncertainty,” in SIGSPATIAL ’12.

[5] J. Yuan, Y. Zheng, L. Zhang, X. Xie, and G. Sun, “Where tofind my next passenger,” in UbiComp ’11.

[6] J. Yuan, Y. Zheng, X. Xie, and G. Sun, “Driving with knowl-edge from the physical world,” in KDD ’11.

[7] S. Standard, “Shenzhen ranks fifth in the world in terms ofpopulation density,” http://www.shenzhen-standard.com.

[8] S. Consulting, “The new york city taxicab fact book,”http://www.schallerconsult.com/taxi/taxifb.pdf.

[9] V. Paxson and S. Floyd, “Wide area traffic: the failure of pois-son modeling,” IEEE/ACM Trans. Netw.

[10] “Taxi-transit integration in the atlanta regio,” in Georgia Re-gional Transportation Authority.

[11] C. Kang, S. Sobolevsky, Y. Liu, and C. Ratti, “Exploring humanmovements in singapore: A comparative analysis based onmobile phone and taxicab usages,” ser. UrbComp ’13.

[12] P. Santi, G. Resta, M. Szell, S. Sobolevsky, S. H. Strogatz,and C. Ratti, “Quantifying the benefits of vehicle poolingwith shareability networks,” ser. Proceedings of the NationalAcademy of Sciences (PNAS), 2014.

[13] D. Zhang, N. Li, Z.-H. Zhou, C. Chen, L. Sun, and S. Li, “ibat:detecting anomalous taxi trajectories from gps traces,” in Ubi-Comp ’11.

[14] W. Liu, Y. Zheng, S. Chawla, J. Yuan, and X. Xing, “Discoveringspatio-temporal causal interactions in traffic data streams,” inKDD ’11.

[15] R. K. Balan, K. X. Nguyen, and L. Jiang, “Real-time trip infor-mation service for a large taxi fleet,” in MobiSys ’11.

[16] H. Gonzalez, J. Han, X. Li, M. Myslinska, and J. P. Sondag,“Adaptive fastest path computation on a road network: a traf-fic mining approach,” in Proceedings of the 33rd internationalconference on Very large data bases, ser. VLDB ’07, 2007.

[17] J. Aslam, S. Lim, X. Pan, and D. Rus, “City-scale traffic esti-mation from a roving sensor network,” in SenSys ’12.

[18] L.-Y. Wei, Y. Zheng, and W.-C. Peng, “Constructing popularroutes from uncertain trajectories,” in KDD ’12.

[19] W. Zhang, S. Li, and G. Pan, “Mining the semantics of origin-destination flows using taxi traces,” in Proceedings of the 2012ACM Conference on Ubiquitous Computing, ser. UbiComp ’12,2012.

[20] J. Yuan, Y. Zheng, and X. Xie, “Discovering regions of differentfunctions in a city using human mobility and pois,” in KDD’12.

[21] X. Zhan, S. Hasan, S. Ukkusuri, and C. Kamga, “Urban traveltime estimation using large scale taxi data with limited infor-mation,” ser. Transportation Research Part C (Emerging Tech-nologies), 2013.

Desheng Zhang (M’10) is a Ph.Dstudent in the Department of ComputerScience and Engineering at the University ofMinnesota-Twin City. His research includesbig data analytics, mobile CPS, wirelesssensor networks, intelligent transportationsystems. He is a member of the IEEE.

Tian He (M’03-SM’12) is an associateprofessor with the Department of Com-puter Science and Engineering, Universityof Minnesota Twin Cities. He is the coauthorof more than 100 papers in premier jour-nals and conferences with more than 12,000citations. As a recipient of the US NSF CA-REER Award’ 09, he served a few programchair position in international conferences.His research includes wireless sensornetworks, intelligent transportation systems,

and distributed systems. He is a senior member of the IEEE.

Shan Lin (M’03) is an assistantprofessor with the Department of Electricaland Computer Engineering in Stony BrookUniversity. He received his PhD in computerscience at the University of Virginia. His re-search is in the area of networked systems,with an emphasis on feedback con-trol based design in cyber physical systems.He works on wireless network protocols,medical devices, and smart transportationsystems. He is a member of the IEEE.

Sirajum Munir (M’13) received hisPhD in Computer Science from Universityof Virginia in 2014. He is currently workingat Bosch Research and Technology Cen-ter as a Research Engineer. His researchinterest lies in the areas of cyber physicalsystems, wireless sensor and actua-tor networks, and ubiquitous computing. Hehas published papers in major conferencesin these areas, two of which were nominatedfor best paper awards at ACM/IEEE ICCPS.

John A. Stankovic (F’94) received the PhDdegree from Brown University. He is the BPAmerica professor in the Computer ScienceDepartment at the University of Virginia. Inthe past, he served as chair of the Depart-ment for eight years. He also won the IEEEReal-Time Systems Technical CommitteeAward for Outstanding Technical Contribu-tions and Leadership. He also won the IEEETechnical Committee on Distributed Pro-cessings Distinguished Achievement Award

(inaugural winner). He has won seven Best Paper awards includingfor ACM SenSys 2006. He was the editor-in-chief for IEEE Transac-tions on Distributed and Parallel Systems. His research interests arein cyber physical systems, distributed computing, real-time systems,and wireless sensor networks. He is a Life Fellow of the IEEE.