+ All Categories
Home > Documents > IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1 RF-3DScan: RFID … · 2019. 10....

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1 RF-3DScan: RFID … · 2019. 10....

Date post: 05-Feb-2021
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
16
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1 RF-3DScan: RFID-based 3D Reconstruction on Tagged Packages Yanling Bu, Student Member, IEEE, Lei Xie, Member, IEEE, Yinyin Gong, Jia Liu, Member, IEEE, Bingbing He, Jiannong Cao, Fellow, IEEE, Baoliu Ye, Member, IEEE, and Sanglu Lu, Member, IEEE Abstract—Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08 of the rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning. Index Terms—RFID, 3D reconstruction, package orientation, package stacking. 1 I NTRODUCTION C URRENTLY, in the logistic industry, traditional applica- tions like the warehouse management and the logistic transportation, are emerging with brand new requirements. For instance, considering the safety and space utilization issues, packages are required to be placed based on specified regulations. Specifically, with regard to a single package, if it contains orientation-sensitive objects, i.e., chemical reagents, precision instruments, it is protected from getting rollover or upside down. While with regard to multiple packages, to ensure the package safety during the transportation process, they are required to be precisely arranged in order, i.e., heavy packages are placed on the bottom and light ones are on the top. To satisfy the above requirements, 3D reconstruc- tion has been introduced to handle these issues for monitor- ing the package placement. Generally, 3D reconstruction is a process of capturing the shape and appearance of a single or multiple real objects. Fig. 1 shows the principle of 3D reconstruction on packaged objects: 1) Package orientation of a single object, which means determining the relative orientation of each object, i.e., pinpointing the bottom/top face and estimating angles of vertical sides of the object in the specified coordinate system. 2) Package stacking of mul- tiple objects, which means determining the relative stacking situation of multiple packages, i.e., figuring out the up- down, front-back or left-right relationships among objects. Yanling Bu, Lei Xie, Yinyin Gong, Jia Liu, Bingbing He, Baoliu Ye, and Sanglu Lu are with the State Key Laboratory for Novel Software Technology, Nanjing University, China. E-mail: [email protected], [email protected], yy- [email protected], [email protected], [email protected], [email protected], [email protected]. Jiannong Cao is with the Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China. E-mail: [email protected]. Lei Xie and Baoliu Ye are the co-corresponding authors. Tag Package Orientation Package Stacking Linear Mobile Scanning Antenna Linear Track Tagged Package above below left right rollover unaligned Fig. 1. 3D reconstruction on tagged packages via linear mobile scanning Previous 3D reconstruction solutions mainly utilize com- puter vision or sensor-based methods. Computer vision- based solutions capture the appearance of objects with cam- eras, and then build 3D profiles of objects [1, 2]. They can reconstruct objects in a vivid way. However, they suffer from the line-of-sight constraint, easily leading to blind angles when capturing images. Sensor-based approaches attach inertial sensors onto items so as to monitor the orientation variation of targets [3, 4]. However, the main disadvantages of them are the high hardware cost and the limited battery life of sensors. Thankfully, the promising RFID technology has brought great chances for the 3D reconstruction on packaged objects in the logistic industry. Nowadays, passive RFID tags have been broadly used to label packages with detailed logistics information. Compared to the above two approaches, the passive RFID tag is battery-free and cheap. Also, RFID technology uses the backscatter communication, so it has no requirement of the line of sight or the light condition. Most importantly, in order to scan and identify packages, RFID systems have been already widely deployed in the sites for most logistic applications in our daily life.
Transcript
  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1

    RF-3DScan: RFID-based 3D Reconstructionon Tagged Packages

    Yanling Bu, Student Member, IEEE, Lei Xie, Member, IEEE, Yinyin Gong, Jia Liu, Member, IEEE,Bingbing He, Jiannong Cao, Fellow, IEEE, Baoliu Ye, Member, IEEE, and Sanglu Lu, Member, IEEE

    Abstract—Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse.Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sightor the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on taggedpackages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signalsfrom the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles foreach tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained packagestacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. Weimplement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show thatRF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08◦ of the rotation angle.For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning.

    Index Terms—RFID, 3D reconstruction, package orientation, package stacking.

    F

    1 INTRODUCTION

    CURRENTLY, in the logistic industry, traditional applica-tions like the warehouse management and the logistictransportation, are emerging with brand new requirements.For instance, considering the safety and space utilizationissues, packages are required to be placed based on specifiedregulations. Specifically, with regard to a single package, if itcontains orientation-sensitive objects, i.e., chemical reagents,precision instruments, it is protected from getting rolloveror upside down. While with regard to multiple packages, toensure the package safety during the transportation process,they are required to be precisely arranged in order, i.e.,heavy packages are placed on the bottom and light ones areon the top. To satisfy the above requirements, 3D reconstruc-tion has been introduced to handle these issues for monitor-ing the package placement. Generally, 3D reconstruction isa process of capturing the shape and appearance of a singleor multiple real objects. Fig. 1 shows the principle of 3Dreconstruction on packaged objects: 1) Package orientationof a single object, which means determining the relativeorientation of each object, i.e., pinpointing the bottom/topface and estimating angles of vertical sides of the object inthe specified coordinate system. 2) Package stacking of mul-tiple objects, which means determining the relative stackingsituation of multiple packages, i.e., figuring out the up-down, front-back or left-right relationships among objects.

    • Yanling Bu, Lei Xie, Yinyin Gong, Jia Liu, Bingbing He, Baoliu Ye,and Sanglu Lu are with the State Key Laboratory for Novel SoftwareTechnology, Nanjing University, China.E-mail: [email protected], [email protected], [email protected], [email protected], [email protected],[email protected], [email protected].

    • Jiannong Cao is with the Department of Computing, The Hong KongPolytechnic University, Hong Kong, China.E-mail: [email protected].

    • Lei Xie and Baoliu Ye are the co-corresponding authors.

    Tag

    PackageOrientation

    PackageStacking

    Linear

    Mobile

    Scannin

    g

    Antenna

    Linear Track

    Tagged Packageabove

    below

    leftright

    rolloverunaligned

    Fig. 1. 3D reconstruction on tagged packages via linear mobile scanning

    Previous 3D reconstruction solutions mainly utilize com-puter vision or sensor-based methods. Computer vision-based solutions capture the appearance of objects with cam-eras, and then build 3D profiles of objects [1, 2]. They canreconstruct objects in a vivid way. However, they suffer fromthe line-of-sight constraint, easily leading to blind angleswhen capturing images. Sensor-based approaches attachinertial sensors onto items so as to monitor the orientationvariation of targets [3, 4]. However, the main disadvantagesof them are the high hardware cost and the limited batterylife of sensors. Thankfully, the promising RFID technologyhas brought great chances for the 3D reconstruction onpackaged objects in the logistic industry. Nowadays, passiveRFID tags have been broadly used to label packages withdetailed logistics information. Compared to the above twoapproaches, the passive RFID tag is battery-free and cheap.Also, RFID technology uses the backscatter communication,so it has no requirement of the line of sight or the lightcondition. Most importantly, in order to scan and identifypackages, RFID systems have been already widely deployedin the sites for most logistic applications in our daily life.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 2

    Therefore, in this paper, we propose a passive RFID-based 3D reconstruction approach, called RF-3DScan. Asshown in Fig. 1, RF-3DScan aims at performing the 3Dreconstruction on packaged objects attached with passiveRFID tags, including the package orientation and the pack-age stacking. The basic idea is that by attaching multipletags onto the surface of packages, we are capable of obtain-ing the orientation of each single package and the stackingstatus of multiple packages based on the backscattered RF-signals from these tags. RF-3DScan works as follows. Weattach a set of passive RFID tags onto the package surface,and leverage one mobile RFID antenna to move along thestraight line to continuously scan the tagged packages. Withthe mobile scanning, we collect RF-signals from tags whenthe antenna is at different positions. Then, we extract phasedifferences of tags at different time points, and build angleprofiles for each tag to depict the geometry angle variationbetween antenna-tag pairs during the moving process. Re-ferring to the angle profiles of tags, we can derive theirrelative positions, and further determine the package place-ment status, including the package orientation for each singlepackage and the package stacking for multiple packages.

    To realize the 3D reconstruction via RFID systems, thereare three key challenges. The first challenge is that theuncertain tag direction is easy to create dead zones of RFIDcommunication. How to optimize the layout of multipletags for avoiding dead zones and achieving the robust 3Dreconstruction is a key problem. To tackle this challenge, wedeploy tags along three mutual orthogonal orientations, sothat there are always some tags that can be collected by thereader easily, which guarantees the high sampling rate andreliable 3D reconstruction. The second challenge is that theexisting work can only derive the 2D relative localizationof tag objects via once mobile scanning. How to locate thepackage and determine the package placement in the 3Dspace is still under-investigated. To tackle this challenge, webuild an angle-profile model and combine this model withthe priori knowledge of tag layout to sense the packageplacement in the 3D space. Through once mobile linear scan-ning, we can extract angle profiles from phase differences toobtain position indicators and further determine the pack-age orientation with the known tag layout. By performingone more scanning along the direction orthogonal to theprevious one, we can combine the twice position indicatorsto accurately estimate the package stacking. Although the2D scanning is a fine-grained solution for the package stack-ing, it requires the extra mobile scanning, so we proposea coarse-grained solution by the 1D scanning. With theknown tag layout, we can localize the package via onlyonce scanning to determine the package stacking. The thirdchallenge is that the RF-signal is likely to be distorted dueto the tag orientation or the multi-path effect. How to selecteffective data to ensure the performance is to be studied. Totackle this challenge, we leverage phase differences to deriveangle profiles, so as to eliminate the phase variation causedby the changing tag’s orientation relative to the antennaduring the mobile scanning. Furthermore, at the start or endof the scanning, as the antenna is relatively far from the tag,the RF-signal is more seriously distorted, so we propose anadaptive algorithm to automatically filter outliers and keepremaining data for the later estimation.

    This paper presents the first study of using RFID toperform the 3D reconstruction on tagged packages. Wemake three contributions. First, for the 3D reconstructionon packages, we attach a set of passive RFID tags ontopackages, and respectively handle issues of the packageorientation and the package stacking through angle profilesof tags. We build an angle-profile-based model to depictthe relationship between RF-signals of tags and the orien-tation/stacking status of packages. Second, we propose amobile scanning approach to perform the 3D reconstructionof tagged packages via RFID. Generally, with the 1D mobilescanning, we can determine the package orientation andcoarse-grained package stacking; while with the 2D mobilescanning, we can determine the fine-grained package stack-ing. Third, We implement a prototype system of RF-3DScanto evaluate its performance. Our experiment results in realsettings show that RF-3DScan can achieve about 92.5%identification accuracy of the bottom face, and average errorabout 4.08◦ of the rotation angle. The 1D scanning is mucheasier to perform than the 2D scanning, while achieving thecomparable performance in terms of the package stacking.

    2 RELATED WORK2.1 Computer Vision and Sensor-based ApproachComputer-vision-based solutions mainly leverage the depthcamera to perform 3D reconstruction of multiple objects[1, 2]. To avoid the blind angles in 3D reconstruction forspecified objects, usually multiple depth cameras are de-ployed at different positions to perform multi-view recon-struction for their 3D models [2], or a moving depth camerais used to build the 3D models in a mobile approach [1]. Ina word, these approaches suffer from the line-of-sight (LOS)constraint in 3D perception, and they are vulnerable to thelimitation of the light intensity. Sensor-based solutions [3, 4]mainly attach the battery-powered sensors (such as inertialsensors or GPS modules) to the surface of the objects, andcontinuously monitor the 3D placement of specified objects,so as to track the orientation variation [3], or the stackingsituation among multiple objects. However, they suffer fromthe high hardware cost of sensors, as well as the limitedbattery life of the sensor.

    2.2 RFID-based ApproachOrientation tracking: By attaching RFID tags onto the spec-ified object, it is possible to track the orientation variationof the object according to the variation of the correspondingRF-signals [5–10]. Tagball [5] is proposed as a 3D human-computer interaction system, where multiple passive tagsare attached to a controlling ball, such that the motionsof the ball from users can be detected from the phasechanges of multiple tags. Tagyro [6] attaches an array ofpassive RFID tags as orientation sensors on the objects,by transforming the runtime phase offsets between tagsinto the orientation angle. Compared with our RF-3DScansystem, these approaches track the orientation variation ofthe dynamically moving objects, whereas our approach aimsto determine the orientation of statically placed packages.

    Localization: RFID localization generally falls into twocategories: absolute localization [11–17] and relative local-ization [18–25]. By attaching multiple tags and pinpointing

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 3

    each tag’s 3D coordinates, the absolute localization can betailored to our problem for 3D reconstruction. However, thisapproach suffers from the complicated system deploymentor the high computational complexity. For example, thestate-of-the-art absolute localization schemes Tagoram [13]and RFind [15] are able to achieve the cm-level localizationaccuracy, however, they require either high computationoverhead or dedicated device calibration, which are unfitfor estimating many packages concurrently. In addition, thelocalization work focuses on pinpointing only a single tag,how to estimate the package placement with the tag arrayis still under-investigated. Rather than the absolute localiza-tion of a single tag, our approach utilizes the tag array toprovide the localization result, which can achieve the com-parable performance without the above limits. Moreover,the relative localization investigates the relative locations ofobjects as opposed to absolute coordinates. STPP [19] is thefirst work to tackle the 2D relative localization. It investi-gates the spatial-temporal dynamics with phase profiles. AV-zone (comprised of phase sequences) based solution isproposed to determine the relative localization of taggedobjects in the 2D plane. However, STPP cannot always geta V-zone in practice, especially when the sampling rate ofper tag is low (due to many tags) or there are some deadzones of the RFID communication. Unlike STPP, regardlessof the V-zone, our work takes full advantages of all phasemeasurements for localization and 3D reconstruction.

    3 ANGLE-PROFILE-BASED MODELINGIn this section, we first discuss the limitations of directlyusing phase values, and introduce how to use the phase dif-ference to model the angle profile for the 3D reconstruction.

    3.1 Limitations of Phase-based Measurement

    The RF phase is a widely used attribute of the wirelesssignal that reflects the phase offset between the receivedelectromagnetic wave and the emitted one, ranging from0 to 2π. Due to the ultra-high working frequency in RFIDsand fine-grained measurement resolution of phase values byCOTS readers, the phase is very sensitive to the distance be-tween the antenna and the tag, which gives us the potentialchance to achieve the accurate 3D reconstruction. Supposes is the distance between the antenna and the tag. Since thebackscatter communication of RFID is round-trip, the signaltotally traverses a distance of 2s in each communication.Besides the distance, some hardware characteristics will alsodistort the phase value. Hence, the phase θ reported by thereader can be expressed as:

    θ =

    (2π

    λ× 2s+ η

    )mod 2π,

    where λ is the wavelength, η represents the phase offsetcaused by the hardware characteristics.

    Although the phase accurately reflects the distance, weface three challenges before putting it into use: 1) The distortfactor η caused by the physical hardware is unknown; 2) Thephase value repeats periodically, it is not feasible to use itdirectly; 3) In addition to s and η, our extensive experimentsshow that the tag orientation influences the phase value θ.

    (a) Rotate along the Z axis

    0 60 120 180 240 300 360

    Rotation (deg.)

    1

    1.5

    2

    2.5

    3

    3.5

    4

    4.5

    Ph

    ase

    (rad

    .)

    (b) Phase change during rotation

    Fig. 2. Measured phase of a single tag rotating along the Z axis

    1

    2

    -. 3

    . × cos2

    )" )&

    4

    56

    .

    Fig. 3. Angle-of-Arrival in staticscanning

    )

    1

    2

    -. 3

    . × cos2

    " &

    4

    5.

    6789:;

    direction

    Fig. 4. Angle-of-Arrival in mobilescanning

    Fig. 2 plots the phase change when a tag rotates along theZ axis. It shows that the phase varies continuously over therotation. Next, we discuss how to use the angle-of-arrivalapproach to overcome above three challenges, and benefitour system design in the sequel.

    3.2 Angle ProfileAngle-of-Arrival (AoA) is one of the most popular RF-basedlocalization measurements using phase difference. The basicidea of our approach is that by moving the antenna to scanthe tags, we extract phase differences from the specified tagsat different time points, then we derive the geometry anglesbetween the tag and the mobile antenna when the antennais at different positions, which is called angle profile.

    3.2.1 Angle in Static ScanningAs shown in Fig. 3, a tag is set at T , A1 and A2 are twoantennas separated by d, M is the middle point of A1A2. Vis the projected point of T on the antenna pair line A1A2,the perpendicular distance is h. The included angle betweenline TM and line MV is the AoA for tag T at position M ,denoted as α. Let dT,A1 and dT,A2 represent the distancesbetween T and the antennas, the antennas collect the phasesas θA1 and θA2 , respectively. θA1 , θA2 ∈ [0, 2π).

    The phase difference is related to the distance differencefrom the tag to the antennas. When h � d, the relationshipbetween the phase difference (∆θ = θA1−θA2+θη , θη meansthe phase offset caused by the hardware characteristics ofA1and A2) and the distance difference (∆d = dT,A1 − dT,A2 'd cosα) can be approximated as:

    2d cosα

    λ=

    ∆θ

    2π+ n, (1)

    where n can be any integer in[− 2dλ −

    ∆θ2π ,

    2dλ −

    ∆θ2π

    ], its

    range is 4dλ . When d <λ4 , the value range of n is smaller than

    1, which means n has a unique value, so α is deterministic.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 4

    Fig. 5. Metrics of the angle profile

    3.2.2 Angle in Mobile Scanning

    As for multiple antennas, the phase offsets related to theirown hardware characteristics are different, so it is hard todetermine θη . Hence, we prefer a mobile antenna to multiplestatic antennas, in which case θη can be canceled.

    For a mobile antenna, the angle-of-arrival is a littledifferent. Without the loss of generality, we redefine the AoAin a mobile case, as shown in Fig. 4. Similarly, T is the tagposition and V is its projected point on the antenna movingline, its perpendicular distance is h. Let the mobile antennabe at position A, then the included angle of line TA and theantenna moving direction is just the angle-of-arrival (α) forthe tag when the antenna is at position A.

    To estimate the angle at position A, we only need thephases collected at the two nearby positions (P1 and P2),centered on the antenna (P1A = AP2). Thus, the phasedifference at position P1 and P2 can be used to estimateα with Eq. (1). By combining the angles at different antennapositions, we can derive an angle profile for a specified tag.

    3.3 Metrics of Angle Profile

    Suppose there are two tags and one antenna in the sameplane (Fig. 5). The antenna moves linearly from O to A,so it passes through T1 first, followed by T2. When theantenna passes through the tag (corresponding to point Vin Fig. 4), the angle-of-arrival (α) of that tag reaches π/2,naming this point as the perpendicular point. Similarly, we callthe distance from the tag to the perpendicular point perpen-dicular distance, the direction perpendicular to the antennamoving direction as perpendicular direction. As T1 is on theleft along the antenna moving direction, its perpendicularpoint shows earlier than T2. Hence, the perpendicular pointis the key metric for the tags’ relative positions along themoving direction.

    Besides the perpendicular point, there is the other specialpoint: equal angle point. The equal angle point is where theantenna and the two tags are in the same line, so T1 andT2 share the same angle. Before equal point, the angle ofT1 is smaller than the angle of T2. On the contrary, theangle of T1 changes to be bigger than that of T2 after theequal angle point. No matter for T1 or T2, its angle increasescontinuously during the antenna moving process, so it isobvious that the angle of T1 changes faster than that ofT2. Such phenomenon is due to the smaller perpendiculardistance of T1. Thus, according to the angle change rate,we can determine the tags’ relative positions along theperpendicular direction.

    Fig. 6. Model of the angle profile

    3.4 Model of Angle ProfileTo depict the angle-profile-based measurement metrics inmathematics, we build a linear model to derive the metricsfrom the angle profile automatically. Considering Fig. 4, theangle-of-arrival can be expressed as:

    cotα =yV − yA

    h, (2)

    where cot means the cotangent function, h is the perpen-dicular distance between the tag and the antenna movingtrace. yA and yV represent the coordinates of point A andV along the antenna moving direction. Assume there is anantenna starting point S, the distance from S to V is l0, theantenna moved distance is l. Thus, (l0 − l) represents thedistance from the antenna to the perpendicular point (sameas (yV − yA)), the angle can be rewritten as:

    cotα = kl + b, k = − 1h, b =

    l0h, (3)

    where the slope k is related to the minus reciprocal of h, theintercept b depends on the ratio of l0 and h.

    Taking the tags in Fig. 5, the transformed angle expres-sion based on Eq. (3) should look like the lines shownin Fig. 6. As l increases continuously during the movingprocess, α increases as well. When the antenna reaches theperpendicular point, α is equal to π/2, so cotα = 0. The lineof T1 reaches 0 earlier than T2. Thus, the order of such zeropoints are corresponding to the tags’ perpendicular points,and the spacing between two zero points just reflects theseparation of tags’ perpendicular points. In addition, theintersection of the two lines represents the position wherethe tags are projected on the same line with the antenna,corresponding to the equal angle point. Specifically, thesmaller h is, the larger ‖k‖ is, and the sharper the line is.As the h of T1 is smaller than T2, the ‖k‖ of T1 is larger, sothe line of T1 decreases faster than T2.

    For a certain tag, its angle profile records its anglesat different positions, as {(cotαi, li)}, i = 1, 2, ..., n, nrepresents the amount of samples. As described in Section3.2.2, it is easy to obtain cotαi at a specific position with theseparation distance and the phase difference of two nearbypositions, here comes a new question, how to determinethe location of the antenna during the moving process?Actually, we only care about the relative position of theantenna along the linear scanning direction when it collectsdata, we need not the absolute position of the antenna inthe 3D space, but we require to know the relative movingdistance along the scanning direction so as to determine thevalue of li in the angle profile. Note that, it is the relativepositions among tags that matters, so we can randomly seta position along the linear scanning direction as the starting

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 5

    point. Then based on the moving distance from the startingpoint, we get the value of li at any time ti. After extractingthe angle profile, referring to Eq. (3), we are able to estimatethe two unknown parameters h and l0 through the linearfitting method. Specifically, h depends on the perpendiculardistance between the tag and the antenna moving trace.l0 is related to the projected position where the antennapasses through the tag, the larger l0 is, the later that linereaches 0, and the tag is more ahead along the antennamoving direction. Thus, by leveraging these properties, wecan determine tags’ relative positions with the followingtwo principles:

    1) The value of ‖k‖ reflects the perpendicular distanceh from the tag to the antenna moving trace: thelarger ‖k‖, the smaller the perpendicular distance.

    2) The value of l0 determines the projected positionof the corresponding tag along the antenna movingdirection. The difference of l0 between two tags indi-cates their interval in the antenna moving direction.

    3.5 Localization based on Angle Profile with Tag ArrayWith angle profiles of each tag, we obtain the perpendiculardistance and projected position along the scanning directionof each tag. The perpendicular distance is related to theposition of the tag, including the height difference and thedepth of the tag from the antenna. Although it is hard todecompose the perpendicular distance to get the heightdifference and depth of each tag, we can localize each tagby taking these tags as a whole, which has the known tagseparation distances along each axes in a certain coordinatesystem. Specifically, taking a tag array with two tags forexample, if the separation distances of two tags along theorthogonal directions of the antenna scanning direction areknown, the positions of the two tags can be obtained basedon their perpendicular distances. As shown in Fig. 7, theantenna moves along the Y axis, the tag separation distancesof T1 and T2 along the X and Z axes are ∆x and ∆z,and the perpendicular distances of T1 and T2 are h1 andh2, respectively. Assume the depth and height differencefrom T1 to the antenna moving trace are dx and dz , theperpendicular distances can be represented as:{

    h21 = d2x + d

    2z,

    h22 = (dx + ∆x)2 + (dz −∆z)2,

    (4)

    where h1 and h2 are extracted from angle profiles of eachtag, ∆x and ∆z are known according to the tag array layoutand the package orientation, so (dx, dz) can be computedusing Eq. (4). Meanwhile, as the projected point of the tagalong the scanning direction is extracted from the angle pro-file, its position along the Y axis is obtained, so the relativeposition of the tag in the 3D space is totally determined.

    4 SYSTEM DESIGN4.1 System OverviewRF-3DScan is a 3D reconstruction system for tagged pack-ages via RFID. In RF-3DScan, the information of each pack-age is priori and stored in the database individually, includ-ing the package size, the position and unique Electronic

    Movingdirection

    Y

    X

    Z01

    02

    ∆4ℎ2

    ℎ1

    6768 ∆9

    Fig. 7. Localization based on the tag array

    Preprocess

    Angle computation

    Linear fitting

    Angle smoothing

    RF

    Signals

    Determine

    package orientation

    (single)

    Determine

    package stacking

    (multiple)

    • Bottom / top face

    • Rotation angle

    Relative stacking

    situation

    Fig. 8. Architecture of RF-3DScan

    Product Code (EPC) of each attached tag. Based on thetag’s EPC from RF-signals, it is easy to obtain the extensivecorresponding package information. Specially, the taggedpackages ought to be produced by machines in the reallogistic industry, that is, the priori knowledge is determinedby the designer, so it is easy to obtain such priori knowledgewithout extra labors for acquisition. Aiming at using asfewer tags as possible to depict the package uniquely, accu-rately and conveniently, the tag deployment should obey thetwo design rules in Section 4.3.1. Meanwhile, we make thefollowing assumptions: 1) The antenna moves at a constantspeed; 2) Each package is a standard cube, and they are fullyon the ground or parallel to the ground (on the ground is aspecial case of parallel to the ground).

    Fig. 8 illustrates the architecture of RF-3DScan. RF-3DScan takes RF-signals from the tags as input, then out-puts 3D profiles for multiple packages. The whole systemconsists of three components: 1) Preprocess: With RF-signalsfrom the tags, RF-3DScan builds angle profiles by usingthe phase differences at different time points for each tag,and extracts position indicators from angle profiles for therelative localization among tags by linear fitting. 2) Deter-mine package orientation for a single package: By comparingthe relative positions of tags on a specified package, RF-3DScan can determine which side of the package is on theground, and then evaluates the angle of the vertical sides ina specified coordinate system. 3) Determine package stackingfor multiple packages: After deriving the orientation of a singlepackage, the centers of the packages are also determinedalong the scanning direction. By performing a 2D mobilescanning, RF-3DScan combines the results from the twoorthogonal scanning, so the accurate relative positions ofthese packages in the 3D space can be determined. Notethat, based on the tag array localization, using only 1Dscanning can also achieve the comparable performance.

    4.2 Data PreprocessWith raw RF-signals, we need to process the collected dataand build angle profiles of tags. The preprocessing can bedivided into three steps: angle computation, angle smooth-ing and linear fitting.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 6

    4.2.1 Angle ComputationAs the antenna collects phases at different time points dur-ing its mobile scanning, we can extract the phase differencesfor a tag at different positions. Using these phase differences,we compute the angle-of-arrivals with Eq. (1). To get adeterministic angle, as mentioned above, the distance of thepositions among two phases should be within λ/4.

    4.2.2 Angle SmoothingAlthough using the phase difference from two positionswith small separation can get a unique angle, the noise likethe multi-path effect would influence the phase measure-ment, there would exist the large fluctuation among angles,so the angle smoothing is required. Usually, the phasecollected by the antenna is not uniform, so is the angle dis-tribution, thus it is not suitable to use the common smoothalgorithms, e.g., low-pass filter. Taking the noise µ intoconsideration: cosα = λ2d

    ∆θ+µ2π +

    nλ2d , when d is very small, µ

    has much influence on cosα. While when d increases, suchdistortion effect decreases, but there exist redundant anglesin the results, only one of them is the true value. Hence, wecan derive two sets of angles from two phase separations: asmall one and a large one, then use the unique angles fromthe small separation to filter the several angle candidatesfrom the large separation, hence, we get a relative accurateangle profile with less fluctuation [18]. Note that, too largeseparation will bring too much environmental change andbreak the restraint of the angle estimation method. Thus,we set the small separation around 5-8cm and the largeseparation within 15cm empirically when the antenna is infront of the packages about 1m.

    4.2.3 Linear FittingWith smoothed angles at different positions from an angleprofile for a certain tag, we can use the linear model asEq. (3) to fit them iteratively, then derive the two importantposition indicators (h and l0) of that tag for the later 3Dreconstruction, details are depicted in Algorithm 1.

    Due to the ambient noise in the environment, the angleprofile contains outliers which may mislead the linear fittingresult, thus we need to figure out these outliers and elimi-nate them to extract accurate position indicators. Denote theangle profile of a certain tag as {(cotαi, li)}, the fitted angleprofile based on Eq. (3) as {( ̂cotαi, li)}. If the absolute angledifference of sample i, i.e., ‖ cotαi − ̂cotαi‖, is larger thanangle difference threshold δm, we take sample i as an outliercandidate. Considering that outliers affect the linear fittingresult, some normal samples may be identified as outliersdue to the inaccurate fitting line, thus we automaticallyset the temporary angle difference threshold based on thedistribution of angle differences. Empirically, we select thelarger value between δm and the biggest decile of the angledifference {δi} as the temporary angle difference thresholdδt, and δm is set to 0.12 in our experiments. Hence, we re-move all outliers whose angle differences are larger than δt.Due to the outliers elimination, the origin sample sequencecan be noncontinuous, i.e., the separation interval betweentwo neighbor samples is larger than a separation intervalthreshold lm. In this situation, we only keep the centralmajor part of the continuous samples, and use these remain-ing samples to redo the linear fitting process until there are

    Algorithm 1 Linear Fitting to Extract Position Indicators.Input:The angle profile of a certain tag, {(cotαi, li)};The threshold of angle difference, δm;The threshold of separation interval, lm;The threshold of sample length, Lm;Output:Position indicators, h and l0;

    1: Do the linear fitting with the angle profile {(cotαi, li)}based on Eq. (3), and get the fitted angle profile{( ̂cotαi, li)}. For each sample i, compute the absoluteangle differences δi = ‖ cotαi − ̂cotαi‖;

    2: while the maximum value of {δi} > δm do3: Calculate the temporary angle difference threshold:δt = max(δm, the biggest decile of {δi});

    4: Remove all outliers with δi > δt in the angle profile;5: Check the continuity of the angle profile, if the sep-

    aration interval between two neighbor samples is largerthan lm, split samples and keep the major continuouspart;

    6: Redo the linear fitting with the remaining samples,and compute the new angle differences {δi};

    7: end while8: if the length of remaining samples > Lm then9: Calculate position indicators h and l0 according to

    the final linear fitting result based on Eq. (3);10: return h and l0;11: else12: Abandon this angle profile;13: return ;14: end if

    0 0.5 1 1.5 2 2.5

    Distance (m)

    -4

    -2

    0

    2

    cot

    Original data

    Original linear fitting

    Processed data

    Processed linear fitting

    Fig. 9. Linear fitting with original data and final processed data

    no outliers. After the linear fitting process, if there are notenough remaining samples, which means angles fluctuateseriously all the time, we abandon such an angle profileof the tag to improve the whole accuracy. Otherwise, wecalculate the position indicators h and l0 from the final linearfitting result, where h represents the perpendicular distancefrom the tag to the antenna mobile trace and l0 reflectsthe projected tag position along the scanning direction. Inour experiments, we set the separation interval threshold as0.6m under the moving speed of 0.12m/s and the samplelength threshold as 200. Taking Fig. 9 for example, theblue original angles contain several outliers and using ouralgorithm can effectively eliminate them at both ends, theremaining yellow angles are much more smoother. Theposition indicators (h, l0) from the original data and the

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 7

    X

    Z

    Y

    X

    (a) Rule1

    Y

    X

    Z

    Y

    X

    (b) Rule2

    Y

    X

    Z

    Y

    ! "

    # $

    %

    &

    (c) Possible solution

    Fig. 10. Rules of deploying tags

    final processed data are (1.4m, 1.42m) and (1.23m, 1.47m),respectively. Compared to ground-truth (1.3m, 1.46m), theaccuracy improves a lot with the outliers elimination.

    4.3 Determine Package Orientation for Each SinglePackageTo reconstruct a single package, it makes the same sense todetermine the package orientation, so we just need to iden-tify the bottom face of this package, and estimate the relativerotation angle of the vertical sides against the antennaplane in a specified coordinate system. Here, the “antennaplane” refers to the extended plane of the antenna’s frontsurface, which should be perpendicular to the ground in oursettings. When the antenna performs the mobile scanninglinearly, it is just in the antenna plane.

    4.3.1 Tag LayoutAiming at determining the package orientation only by once1D mobile scanning, we need to deploy tags in an efficientand robust way. The design principle of tag deployment is touse as fewer tags as possible to depict the package uniquely,accurately and conveniently. Specifically, as the packagecan be with any orientation in the 3D space, we shouldpay attention to ensuring there are always enough effectivetags reflecting the signals to the antenna. Meanwhile, as foridentifying the bottom face of the package, it is the same tofind which tags are along the vertical axis and what orderthese tags are with. Therefore, we make two rules as follows.

    1) The orientations of tags should be along different orthog-onal axes (in Fig. 10(a)). With this rule, tags along onedirection at most are in the blind direction, so othertags can response to the antenna successfully.

    2) Tags should be deployed along different orthogonal axes(in Fig. 10(b)). With this rule, whatever the orienta-tion of the package is, there is always one tag pairalong the vertical axis.

    Combining the above two rules, Fig. 10(c) illustratesa possible tag deployment satisfying these two rules. Nomatter what orientation the package is, there are four tagsat least to avoid the signal blind direction. Also, there isalways one tag pair along the Z axis. By determining thetags’ order of this tag pair, we can derive which side of thepackage is on the ground then.

    4.3.2 Linear Fitting of Tags with Different OrientationsConsidering that tags in the layout as Fig. 10(c) havedifferent orientations, we carry out experiments to checkwhether different orientations among tags have effects onthe accuracy of the linear fitting. As shown in Fig. 11,we arrange six tags with three orientations as required in

    0.5 m

    0.3 m

    0.5 m0.3 m

    1 m

    Linear Scanning

    !"

    !# !$

    !%

    !&

    !'

    Fig. 11. Experiment setup of linear fitting accuracy with different tagorientations

    T1-T2 T3-T4 T5-T6 T1-T3 T3-T50

    1

    2

    3

    4

    Err

    or

    (cm

    )

    Fig. 12. Accuracy along scanningdirection

    T1 T2 T3 T4 T5 T60

    5

    10

    15

    Err

    or

    (cm

    )

    Fig. 13. Accuracy along orthogo-nal direction of scanning

    Fig. 10(c), the results are plotted in Fig. 12 and Fig. 13.We observe that the accuracy of projected position l0 alongthe scanning direction keeps stable with different orientations,while the perpendicular distance h does not have the same highaccuracy as l0. Fig. 12 plots the accuracy of l0 along theantenna scanning direction, i.e., the absolute separationerror between two tags along the scanning direction. Mostof the absolute separation errors of different tag pairs arebelow 1.8cm, even in the worst case, the separation achievesaround 92.3% accuracy. Fig. 13 plots the accuracy of theperpendicular distance h along the orthogonal direction tothe scanning. The error is much larger than the separationerror along the scanning direction. For tags with the sameorientation, the near-far relationships between tags and theantenna are reliable, but for tags with different orientations,the error may incur the wrong near-far relationship, i.e., T3has the smaller h than T2, but T3 has the larger estimated h.Therefore, when using position indicators, we can leveragethe projected positions of tags with different orientations,and prefer the perpendicular distances of tags with the sameorientation for comparison or calculation.

    4.3.3 Determine Package Orientation

    To determine the package orientation, it demands to identifythe bottom face of the package and the relative angle of thevertical sides in a specified coordinate system. Consideringour assumption that one side of the cube package must beparallel to the ground, when we deploy tags like the solutiondescribed in Fig. 10(c), we can identify which tag pair isalong the Z axis and what order the tag pair is in fact. Inthis case, let the antenna do the mobile scanning along theX or Y axis only once, we can determine the orientation fora single package. Note that, the antenna should be aboveor below all the tags, and the package is at the same sideagainst the antenna plane during the scanning process.

    For simplicity, suppose the antenna is above all the tagsand ahead of all the tags along the X axis positive direction,it moves along the Y axis, then the tag pair along the Zaxis is perpendicular to the antenna scanning direction, so

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 8

    !

    "

    !

    "

    X

    Z

    Y

    # $

    %

    &Y

    X

    (a) Possible case1

    𝑇"

    𝑇#

    X

    Z

    Y

    𝑇' 𝑇(

    𝑇)𝑇*

    𝑇' 𝑇(X

    Z

    Y𝑇*𝑇)

    𝑇#

    𝑇"

    X

    Z

    Y𝑇' 𝑇(

    𝑇"𝑇#

    𝑇)

    𝑇*𝑇' 𝑇(

    𝑇#𝑇"X

    Z

    Y𝑇*

    𝑇)

    (b) Possible case2

    # $ # $ X

    Z

    Y &

    %

    "

    ! Y

    (c) Possible case3

    Fig. 14. Possible cases for the special package orientation

    their perpendicular points should be the same. That is, thespacing of their perpendicular points equals 0 in theory.Hence, if the spacing of the perpendicular points for a tagpair is 0, it is probable that the tag pair is along the Zaxis now, except for some special cases where there is atag pair along the X axis, then there are two tag pairs thattheir perpendicular points spacings are equal to 0, we willdiscuss it later. After identifying which tag pair is alongthe Z axis, we can determine the tags’ order by comparingtheir perpendicular distances extracted from their angleprofiles. As the antenna is above all the tags, the tag withthe smaller perpendicular distance of the vertical tag pairshould be above the other along the Z axis. However, suchcomparison ignores the relationships of the perpendiculardistances for other tags, it is easy to make a wrong decisionwith only one comparison results. Note that, the spacings ofthe perpendicular points of the tags should stay the samewhen the package is upside down, as the package rotatesaround the Y axis by 180◦. So we can estimate the relativeangle of the package first, then use the relationships of theperpendicular distances among different tag pairs to votefor the tags’ order of the vertical tag pair, then determinethe bottom face of the package. When selecting the tagpairs among all the tags on the package, it is significantto avoid the tags in the blind direction by filtering the tagswith relative weak RSS compared with other tags on thepackage. The angle estimation is based on the spacings ofthe perpendicular points for different tag pairs, as:

    arg minφ

    N∑i=1

    ‖δ′i − δi(φ)‖, (5)

    where N is the number of tag pairs, δ′i is the measuredspacing between the perpendicular points of a tag pair, δi(φ)represents the spacing at relative angle φ theoretically.

    Now, considering the case shown in Fig. 10(c), we il-lustrate how to deal with the special cases where there aretwo tag pairs whose perpendicular point spacings are bothequal to 0. As the antenna moves along the Y axis, theperpendicular points of the tag pair on the same surface{T3, T4} or {T5, T6} are at the same point. With the relativeorder of the tag pair {T1, T2}, as the tag T1 is on the left of T2along the antenna moving direction, there are four possiblecases of the package orientation, as shown in Fig. 10(c) andFig. 14. Any of these possible cases can transform into an-other case by rotating along the Y axis, but the relationshipsof their perpendicular distances differ, so we can use theserelationships to vote for which case is the most possible case.As we assume that the antenna is above all the tags andahead of all the tags along the X axis positive direction,let the perpendicular distances for T1, T3, T5 be h1, h3, h5,then for the case as Fig. 10(c): h1 < h3, h3 > h5, for thepossible case1 (Fig. 14(a)): h1 < h3, h3 < h5, for the possible

    case2 (Fig. 14(b)): h1 > h3, h3 < h5 and for the possiblecase3 (Fig. 14(c)): h1 > h3, h3 > h5. There are multiple tagpairs for the comparison, here we list part of them for ex-planation. Then, by comparing the relationships of differenttag pairs, we vote for the possible cases, and select the casewith the highest score as our estimation result. Consideringthat different tags have different credibilities as they areinfluenced by the ambient noise to different extents, whenvoting for the relationships, we tend to allocate the heavierweight to the tag pair with the better linear fitting, such that,we set the voting weight of a tag pair with the reciprocalaverage linear fitting error of the two tags. Moreover, if thelinear fitting error is large than the given threshold, that tagwill be abandoned in the following analysis. Through theweighted voting, we can effectively avoid the ambiguoussituation that several relationships get the same scores so asto improve the accuracy of the orientation estimation, thecorresponding experiments are referred to Fig. 28.

    4.3.4 Discussion1. There must be a side of the package parallel to the ground.As we assume that there must be a side of the packageparallel to the ground (which means the package is on theground or on other packages, not tilted), thus the state ofthe package is limited, the angle estimation is restricted toalong the Z axis. If not, the searching space of finding theoptimal angle expands, the simple solution is to add onemore mobile scanning along the direction different from theprevious one, the 3D reconstruction can be realized as well.

    2. There may exist serious coupling effect and interrogationfailure when packages are stacked closely. As packages arestacked closely in the warehouse, the large amounts of tagsand small separations between tags from different packagesmay cause serious coupling effect or interrogation failureproblems [26–30]. Such problems can have great influenceon the robustness of our proposed solution, which are reallydifficult to deal with only through the advanced algorithms,the improvement of physical tag design is fundamental andplays a more important role in solving these problems. Theadvanced algorithms can mitigate these problems, but it isnearly impossible to solve the problems thoroughly. Thegoal of this paper is to propose a scheme for 3D reconstruc-tion on tagged packages, so we do not put much effort intodealing with these problems, instead we design a prototypesystem to show the idea with the relative ideal environment,where the coupling effect and multi-path are not so serious.Moreover, we still try to explore the effect of coupling effectdue to the small tag separation on the robustness, details areshown in Section 5.2.2 and Fig. 24. Previous work Tagyro [6]does researches on the coupling effect between neighboringtags, and finds that when there exists strong coupling effect,the actual tag separation to the antenna is not the same asthe physical tag separation, so it is necessary to measurethe actual tag separation for the following processing. Theyprovides the measurement of the actual tag separation to theantenna, which is likely to be utilized in our scenario. Withthe actual tag separation, the performance of our solutioncan be improved obviously.

    3. The difference of the perpendicular distances for the tagpair parallel to the antenna plane may be much smaller thanthat for the tag pair perpendicular to the antenna plane with

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 9

    -./

    -.01" 1&

    1'

    Fig. 15. The distance differences are differ-ent for the equal tag separation along differ-ent directions

    )

    *

    +

    ,

    )

    *

    +

    ,

    )

    *

    +

    ,

    YX

    Z

    Fig. 16. Different kinds of package stacking

    Moving direction

    Y

    X

    Z

    !

    "

    # $

    %

    &

    '

    (

    Fig. 17. Determining package stacking in 3D space

    the same tag separation. As shown in Fig. 15, the tag pair{T1, T2} is perpendicular to the antenna plane, while thetag pair {T2, T3} is parallel to the antenna plane. Their tagseparation distances are the same, as ∆dh = ∆dv , but theirperpendicular distance differences are not similar. Supposethe distance from T1 to the antenna is 1m, the tag separationis 0.2m, so the perpendicular distance difference betweenT2 and T3 is only 1.65cm, which is much smaller than thatbetween T1 and T2 (20cm). Since the distance difference isso small, the relationship of such tag pair is probably to bewrong. So, besides the fitting effect, it is better to take intoaccount the perpendicular distance differences of a tag pairwhen setting weights for voting package orientations.

    4. The rule of deploying tags in Section 4.3.1 is a simplesolution but not a unique solution. The rule that the orien-tations of tags should be along different orthogonal axes(in Fig. 10(a)) is to ensure the high probability of acquiringsufficient effective backscattered signals. Actually, it is notnecessary to restrict the tag orientations along axes, anytag orientation is fine as long as ensuring the probabilityof successful tag interrogation regardless of the packageplacement. Meanwhile, as mentioned in Section 4.3.2, thetag orientation has impacts on the linear fitting result, espe-cially for the perpendicular distances of tags with differentorientations. If two tags should have the same orientationin design but have different orientations in reality causedby the poor tag attachment, it is probable to degrade theperformance. What’s worse, if the tag array is wronglystored in the database, i.e., the EPC of a tag is wronglyrecorded, or the tag array is attached to the wrong box,such bad situations are beyond the ability of our solution.However, if applied to the practical industry, the taggedboxes should be automatically produced by machines, thetag attachment ought to satisfy the production standards, soin this paper, we do not bring in the poor tag attachment orother exceptions deliberately.

    5. The antenna has to move along an absolutely straight lineand knows its real-time moving distance. Our angle-profile-based model has a basic requirement that the antenna hasto move along a straight line, such that we can build theangle profile and then use it to extract position indicatorsby referring to Eq. (3). If it happens that the antenna doesnot move along an absolutely straight line, it would causeestimation errors of the antenna’s moving distance andfurther degrade the performance. To handle with such apractical problem, one possible solution is to fix the antennaon a linear mobile track as shown in Fig. 1. Also, it is likely todesign solutions to compensating the location error becauseof the imperfect linear movement in the future. Note that, itwill cost much more to localize an antenna than to move it

    with a constant speed, so in our work, we assume that theantenna can perform the mobile scanning with a constantspeed v. As a result, it is easy to get the moving distancel during the time difference ∆t, as: l = v × ∆t. It wouldbe better that the antenna can localize itself accurately, butour assumption is more economical and convenient to besatisfied in practice.

    4.4 Determine Package Stacking for Multiple Packages4.4.1 Limitations of Position Indicators for DeterminingPackage StackingWhen we determine the package orientation, we derive theindicators for the relative positions among the tags on asingle package. As the geometry relationships of these tagsare known, we can combine these indicators (perpendiculardistance and perpendicular point) from different tags toestimate the indicators of the package’s center point. Then,similarly, we compare the indicators of the center pointsof different packages to determine their stacking situation.Note that, these indicators extracted from once 1D scanningmay not support the package stacking determination due tothe 3-DoF in the 3D space. As shown in Fig. 16, supposethe antenna is above all the packages and is in front of thepackages along the X axis, it moves along the Y axis. Itis easy to determine the package orders along the Y axisby referring to their perpendicular points, but it may be aproblem to determine their orders in the XZ plane. If thepackages line up, that is, the packages are along the Z axisor along theX axis, as the left two cases shown in Fig. 16, wecan determine their orders along their lining up direction bycomparing their perpendicular distances of their centers. Forinstance, the perpendicular distance of package A should besmaller than others, so packageA is ahead of other packagesalong the X/Z axis. If not, however, as the 2 × 2 packagestacking of the right case in Fig. 16, we cannot identify theorders of the packages exactly along the X and Z axes at thesame time only through the position indicators. To solve thisproblem, our basic solution is to perform one more mobilescanning along the orthogonal direction of the previousscanning direction, so as to limit the number of the freedimensions in the 3D space. The more times of the mobilescanning is, the much more the cost will be, thus we adoptthe mobile scanning twice as the least needed times. Suchthat, the package stacking can be determined in the 3D spacewith a 2D mobile scanning. Note that, it is much easier andcosts less to perform the 1D scanning than the 2D scanning,and in some circumstances, we are only able to perform the1D scanning, so based on the localization of the tag array,we also propose a contemporary solution to determining thepackage stacking with only once 1D scanning.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 10

    Z

    X

    Z

    & ' # (

    " % $ !

    *" *&

    (a) Along the X axis

    Y

    Z

    $ # ( !

    " & ' %

    )" )&X

    X

    (b) Along the Y axis

    Y

    X

    Y

    # ()"

    )& & ' " %

    $ !

    *" *&

    (c) Split plane

    Y

    Y

    X

    ZZ

    (d) Split space

    Fig. 18. Illustration of determining the packages’ order in the 3D space

    4.4.2 Determine Package Stacking with 2D Scanning

    Through once 1D scanning, after determining the packageorientation for each single package, we derive the indicatorsof the packages’ centers for the relative positions along thescanning direction, so we know their relative orders alongthat direction. Similarly, by performing the other scanningalong the orthogonal direction of the previous one, we canget the packages’ relative positions along the new direction.Combining their relative positions along the two directions,the 3D space is divided into many pieces. For each piece,two dimensions are fixed, the packages in it are lining up,so we can use the perpendicular distances of the packages’centers to determine their orders in the piece. Thus, all thepackages’ relative positions in the 3D space are determined.

    Taking the scene in Fig. 17 for example, there are eightpackages in total (using the package center to represent thecorresponding package). The dash lines are parallel to thedifferent axes, respectively. The antenna is above all tags. Itperforms a 2D mobile scanning along the X and Y axes. Allof the tags are always at the same side against the antennaplane. For the scanning along the X axis, based on the per-pendicular points of the packages’ centers, the packages canbe split into two sets: {P2, P3, P6, P7} and {P1, P4, P5, P8}.In each set, the X coordinates of the packages are the same,as they share the same perpendicular points projected tothe X axis. Similarly, based on the mobile scanning alongthe Y axis, the packages can also be split into two sets:{P5, P6, P7, P8} and {P1, P2, P3, P4}. Combining these twosplit results, we have four sets then: {P1, P4}, {P2, P3},{P5, P8} and {P6, P7} (in Fig. 18). In each set, the packagesshare the same coordinates along the X and Y axes. Then,we just need to determine the packages’ orders along theZ axis. By referring to the perpendicular distances of thepackages’ centers, we can identify the packages’ orders ineach piece. As we determine the relative positions of eachpiece, and the packages’ relative positions in each piece, thestacking situation of these packages are determined, the 3Dreconstruction for multiple packages is done.

    TABLE 1Comparison of 1D scanning and 2D scanning

    1D scanning 2D scanning

    MethodCoordinatesbased onlocalization

    Height: Average perpendiculardistanceDepth: Perpendicular points alongtwo orthogonal directions

    Setup 1D scanning is much easier to deploy and costs less

    Performance 2D scanning provides more accurate alignmentestimation and is more robust

    4.4.3 Determine Package Stacking with 1D ScanningBy doing once 1D scanning, we can get the package orien-tation, and due to the known tag layout, the tag separationdistances of any two tags along axes are determined. Suchthat, we can leverage the tag array to localize the packagewith the model proposed in Section 3.5. Considering theambient noise and the multi-path effect, it is likely that wecannot solve Eq. (4) with only two tags. So we combinesignals from all effective tags and utilize Minimum MeanSquared Error (MMSE) method to get the optimal solutionto Eq. (4). Specifically, denote the extracted perpendiculardistance of tag Ti from the angle profile as ĥi, and thetheoretical perpendicular distance as hi, which can be easilyderived based on Eq. (4). Thus, we aim to find the optimalsolution (d∗x, d

    ∗z) to minimize the difference between the

    theoretical hi and the extracted ĥi:

    arg mindx,dz

    n∑i=1

    (hi − ĥi

    )2,

    where n represents the number of selected tags. We prefertags with the same orientation or with relatively small linearfitting errors for localization.

    With comparison to the 2D scanning (TABLE 1), the 1Dscanning is not as accurate or robust as the 2D scanning.The perpendicular point accuracy along the scanning di-rection is more accurate than the perpendicular distanceaccuracy along the orthogonal direction of the scanning (inSection 4.3.2). For the 1D scanning, it uses the perpendiculardistances of tags in a tag array to do the localization, sothe error in the estimated perpendicular distance can beintroduced into the height and depth estimation. Mean-while, it is possible that there is no solution to Eq. (4), inwhich case we cannot determine the package stacking basedon the localization. Whereas, the 2D scanning performsone more mobile scanning along the orthogonal direction,the depth depends on the perpendicular point along thesecond scanning direction, which is of high accuracy so asto provide the accurate alignment estimation. The space canbe split by the combination of accurate perpendicular pointsalong two directions, and we use the average perpendiculardistance of each package in a stack to determine their up-down relationship, which is usually robust. However, the1D scanning is much easier to deploy and costs less to per-form. According to our experiment results (in Section 5.2),1D scanning achieves the comparable performance.

    4.4.4 Determine Complex Package StackingIn the above sections, we simplify the stacking determina-tion by taking packages as individual points. However, in

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 11

    1st Linear Scanning

    2 ndLinear Scanning

    Antenna

    ReaderMoving Car

    Tagged Boxes Tags

    Fig. 19. Deployment of RF-3DScan

    a practical warehouse, there are several sizes of packages,it is possible that several small packages are stacked on abig package. If we still view packages as points, we cannotdetermine such complex stacking. Therefore, to handle withthe complex situation, after determining the orientation ofthe tag array, it is necessary to reconstruct the package in the3D space based on the priori knowledge, i.e., the geometricrelationship of the tag array attached on the package. Then,we can determine the complex stacking by comparing theprojected polygons of packages in different planes. Specif-ically, by comparing the relationship of projected polygonsof packages in the XY plane, i.e., parallel to the ground,we can determine which packages are likely to be stackedtogether. Then, by comparing the relationship of projectedpolygons of packages in the XZ and Y Z plane, we candetermine the left-right and up-down relationship amongpackages in a complex stack.

    5 PERFORMANCE EVALUATION5.1 Experiment Settings

    We build a prototype system of RF-3DScan, as shownin Fig.19. Hardware: Our system consists of one ImpinJSpeedway R420 reader, one Laird S9028 RFID antenna andmultiple ImpinJ E41-B tags. The antenna is fixed on a mov-ing car with the height of about 1.8m. Software: We adoptLLRP protocol to communicate with the reader, and usea special module to control the movement of the car. Ouralgorithms are implemented in MATLAB and the languageJava. Deployment: We let the antenna be above the boxesand move at a constant speed of 0.12m/s. For diversity,we use three different sizes of boxes. The sizes of box1,box2 and box3 are 0.4m×0.35m×0.33m, 0.35m×0.27×0.4m,0.49m×0.39m×0.32m, respectively. For each box, there aresix tags on it as shown in Fig. 10(c), the tag separation oftwo tags on the same surface is the same, as 0.23m, 0.17mand 0.2cm for box1, box2 and box3, respectively.

    5.2 Micro-Benchmarks

    Metrics: To evaluate the package orientation accuracy, we settwo metrics: bottom face accuracy and angle error. The bottomface accuracy is defined as the number of the packageswhose bottom faces are identified correctly out of the totalpackage number. The angle error is the error between theestimated angle of the vertical faces against the antennaplane and the actual angle. For the package stacking, weuse the metric ordering accuracy. The box is ordered correctlyonly when its detected order is same as the actual order.

    5.2.1 Orientation Accuracy with Different Scanning Rangeswith or without Outliers EliminationThe most advantage of our approach compared to STPP [19]is that we do not require the large range scanning. We putbox1 in front of the antenna plane about 0.8m, and adjustscanning ranges from 0.3m to 0.9m on a single side or bothsides. Taking range of 0.3m for example, for a single side, itmeans the antenna starts scanning from the box and moves0.3m. While for both sides, the scanning range is 0.6m,centered on the box. To validate the efficiency of outlierselimination during the linear fitting process, we first useall data to do the linear fitting, then eliminate outliers forcomparison. The results without the outliers elimination areshown in Fig. 20-21. From Fig. 20, RF-3DScan achieves theaverage bottom face accuracies about 95% for both sidesscanning, and about 70% for one side scanning range of0.7m. When the both scanning range achieves 0.5m, RF-3DScan can identify the bottom face totally correctly. Forthe single side scanning, RF-3DScan fails to work with thescanning range less than 0.5m. With the range extendingto 0.7m, the accuracy increases to 70%. From Fig. 21, theaverage angle error for the both sides scanning is smallerthan that for the single side scanning. Also, the angle errorwill not be affected by the scanning range significantly.Though the accuracy of the one side scanning is not thatgood, STPP cannot even deal with such limited scanningranges. The similar results with the outliers eliminationare shown in Fig. 22-23. With the outliers elimination, theobvious improvement is obtained when the scanning rangeis larger than 0.7m.

    5.2.2 Orientation Accuracy with Different Tag SeparationDistancesWe adjust the tag separation of box3 from 11cm to 23cm,which is in front of the antenna plane about 1m, and letthe antenna perform both sides scanning of 0.5m 40 timesfor each separation. As shown in Fig. 24, with the largertag separation, the bottom face accuracy gets higher andthe angle error gets smaller correspondingly. It is foundthat when the tag separation is no less than 17cm, theperformance tends to be stable, and we can identify thebottom face of the box accurately with the angle errorsmaller than 3.05◦. While when the tag separation is smallerthan 17cm, the accuracy is seriously affected by the changeof tag separation. According to previous work Tagyro [6],when the tag separation is larger than 15cm, the couplingeffect between tags is relatively small, while for the smallertag separation, the coupling effect gets stronger very fast,which agrees with our experiment results. The couplingeffect can be handled with the method proposed by Tagyro,which is beyond the scope of this paper, so we choose thetag separation larger than 17cm to limit the coupling effect.

    5.2.3 Orientation Accuracy with Different ContentsIn order to evaluate the effect of items made of different ma-terials in the box, we fill box3 with items made of differentmaterials, like plastic toys (Pla), cotton pants (Cot), foam(Foa), mineral water (Liq), antennas made of plastic andmetal (Mix), or just let the box empty (Emp) for comparison.We perform the scanning with the same settings as above.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 12

    0.3 0.4 0.5 0.6 0.7 0.8 0.9

    Scanning Range (m)

    0

    0.2

    0.4

    0.6

    0.8

    1A

    ccu

    racy

    Both sides

    Single side

    Fig. 20. Bottom face accuracy fordifferent scanning ranges withoutoutliers elimination

    0.3 0.4 0.5 0.6 0.7 0.8 0.9

    Scanning Range (m)

    0

    5

    10

    15

    An

    gle

    Err

    or

    (deg

    .)

    Both sides

    Single side

    Fig. 21. Angle error for differentscanning ranges without outlierselimination

    0.3 0.4 0.5 0.6 0.7 0.8 0.9

    Scanning Range(m)

    0

    0.2

    0.4

    0.6

    0.8

    1

    Acc

    ura

    cy

    Both sides

    Single side

    Fig. 22. Bottom face accuracyfor different scanning ranges withoutliers elimination

    0.3 0.4 0.5 0.6 0.7 0.8 0.9

    Scanning Range(m)

    0

    5

    10

    15

    An

    gle

    Err

    or

    (deg

    .)

    Both sides

    Single side

    Fig. 23. Angle error for differ-ent scanning ranges with outlierselimination

    11 14 17 20 23

    Tag Separation (cm)

    0

    4

    8

    12

    16

    20

    Angle

    Err

    or

    (deg

    .)

    0

    0.2

    0.4

    0.6

    0.8

    1

    Acc

    ura

    cy

    Fig. 24. Orientation accuracy fordifferent tag separation distances

    Emp Pla Cot Foa Liq Mix

    Different Contents

    0

    4

    8

    12

    16

    20

    Angle

    Err

    or

    (deg

    .)

    0

    0.2

    0.4

    0.6

    0.8

    1

    Acc

    ura

    cyFig. 25. Orientation accuracy fordifferent contents

    1 1.5 2 2.5 3 3.5 4

    Tag-antenna Distance (m)

    0

    0.2

    0.4

    0.6

    0.8

    1

    Acc

    ura

    cy

    1 set2 sets

    Fig. 26. Bottom face accuracy fordifferent tag-antenna distancesand multiple tag arrays

    1 1.5 2 2.5 3 3.5 4

    Tag-antenna Distance (m)

    0

    5

    10

    15

    20

    Angle

    Err

    or

    (deg

    .)

    1 set2 sets

    Fig. 27. Angle error for differenttag-antenna distances and multi-ple tag arrays

    The results are shown in Fig. 25. It is found that the contentsmade of plastic, cotton or foam have little influence on theaccuracy. Their accuracies are very close to the accuracywhen the box contains nothing, as the bottom face accura-cies are all above 97% and the angle errors are around 2.7◦-3.2◦. However, the accuracies of mineral water and antennasdecrease obviously. Specifically, the bottom face accuracy ofmineral water is about 72.5%, the angle error of antennasis about 8.5◦. The reason is that the liquid and metal havestrong absorption and reflection to signals, when the boxcontains contents like them, the signals of outside tags onthe box can be distorted due to the absorption and reflection,so the accuracy decreases correspondingly.

    5.2.4 Orientation Accuracy with Different Tag-antenna Dis-tances and Multiple Tag Arrays

    We adjust the distance between the antenna plane andbox3 (abbreviated as tag-antenna distance) from 1m to 4m,and perform the scanning with the same settings as above.Denote the tag layout shown in Fig. 19 as one set of tag array.According to Fig. 26-27, for the default one set of tag array,with the increasing tag-antenna distance, the bottom faceaccuracy keeps decreasing and the angle error is increasing.Note that, the accuracy decreases faster when the distancegets larger, i.e., the bottom face accuracy decreases by 10%and 25%, and the angle error increases by 0.62◦ and 4.49◦

    from 1m to 1.5m and from 2.5m to 3m, respectively. Thereason is that with the larger distance, the signals directlytransmitted to the tag from the antenna get weaker muchfaster. Relatively, other noisy signals play a more importantrole in the received signals by the reader, so the signal-to-noise ratio of the received signals decreases very faster,incurring the sharp accuracy decreasing. Meanwhile, weadd one set of tag array on the other three surfaces of thebox, avoiding the serious coupling effect among tags, toexplore the effect of multiple redundant tags. It is found

    that the two sets of tag array perform better than one set.When using the default one set of tag array, the maximumtag-antenna distance is 2.5-3m to ensure the accuracy above60%, while using two sets of tag array, the maximum tag-antenna distance extends to 3-3.5m to ensure the accuracyabove 60%. It is reasonable as we can have more choicesof tag pairs for comparison, so it is probable that morereliable tag pairs are selected for estimation. Despite of thesmall improvement of the monitoring depth when ensuringthe same accuracy, it shows the opportunities to improvethe performance with more redundant tags. Note that, al-though the performance seems barely satisfactory, we canuse drones with portable antennas to replace the lineartrack, such that we are able to design more flexible scanningroutes and further enlarge the monitoring area to cover thewhole warehouse.

    5.2.5 Orientation Accuracy in Package Stacking Situation

    We change the number of boxes from two to four, put allboxes in a stack in front of the antenna plane about 1.5m,and let the antenna perform both sides scanning of 0.5m40 times for each stack. When testing on only one taggedbox as above, we usually obtain signals of all tags withoutmuch noise. While for the package stacking situation, asmany boxes are in a stack, the close distance between taggedboxes can lead to the tag missing or many outliers due tothe heavy multi-path effect. With fewer effective tags, thereare fewer tag pairs for selection to vote for the orientation. Ifall tag pairs have the same weight to vote, it is easy for twoorientations to get the same score. Fig. 28 plots the bottomface accuracy for the box stack with different numbers ofboxes affected by voting weights of tag pairs. Throughsetting voting weights, the accuracy improves significantly,i.e., for the two-box stack, the accuracy improves from 70%to 92.5%. Also, the bottom face accuracy gets smaller withthe increasing number of boxes. It is reasonable that as more

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 13

    2 3 4

    Number of Boxes

    0

    0.2

    0.4

    0.6

    0.8

    1A

    ccura

    cy

    No weight

    Weight

    Fig. 28. Bottom face accuracy forpackage stacking

    2 3 4

    Number of Boxes

    0

    1

    2

    3

    4

    5

    Angle

    Err

    or

    (deg

    .)

    Fig. 29. Angle error for packagestacking

    2 3 4

    Number of Boxes

    0

    0.2

    0.4

    0.6

    0.8

    1

    Ord

    erin

    g A

    ccura

    cy

    1D

    2D

    Fig. 30. Ordering Accuracy forpackage stacking

    2 3 4

    Number of Boxes

    0

    0.2

    0.4

    0.6

    0.8

    1

    Pro

    bab

    ilit

    y

    Fig. 31. Probability of successfullocalization estimation

    0 10 20 30

    Localization Error (cm)

    0

    0.2

    0.4

    0.6

    0.8

    1

    CD

    F

    l0

    Depth

    Fig. 32. Localization accuracyalong orthogonal directions

    Box Distance0

    0.2

    0.4

    0.6

    0.8

    1A

    ccu

    racy

    RF-3DScan

    STPP

    Fig. 33. Bottom face accuracy fordifferent cases

    0 5 10 15 20 25

    Angle Error (deg.)

    0

    0.2

    0.4

    0.6

    0.8

    1

    CD

    F

    RF-3DScan

    STPP

    Fig. 34. CDF of angle error fordifferent boxes

    0 4 8 12 16

    Angle Error (deg.)

    0

    0.2

    0.4

    0.6

    0.8

    1

    CD

    F

    RF-3DScan

    STPP

    Fig. 35. CDF of angle error fordifferent distances

    boxes are put together, the heavier multi-path effect willdecrease the number of effective tags, so the accuracy getssmaller. Different from the bottom face accuracy, as shownin Fig. 29, the angle error does not change much with thenumber of stacking boxes, which is around 4.5◦.

    5.2.6 Stacking AccuracyWe analyze the ordering accuracy for the package stack-ing with the same package stacking settings as above. Todetermine the package stacking, through the 2D scanning,we split the space and then determine the up-down rela-tionship of boxes in the same stack based on the averageperpendicular distances. While through the 1D scanning,we localize packages so as to split package stacks andfurther determine the up-down relationship of packages inthe same stack based on the coordinates along the verticaldirection. Fig. 30 plots the ordering accuracy with differentkinds of scanning. We find that using the 2D scanning cantotally accurately determine the package stacking of two orthree boxes, with the more boxes in the stack, the orderingaccuracy decreases but is still above 89%. Using the 1Dscanning can achieve the comparable performance of the2D scanning. When the number of boxes gets larger, thedifference of the ordering accuracy between the 1D scanningand the 2D scanning becomes larger, but the worse accuracyis still above 80%, around 89% of the accuracy with the2D scanning. The main reason why the 1D scanning is lessrobust than the 2D scanning is that the probability of doingthe localization with the 1D scanning decreases with theincreasing number of boxes. In the stacking situation, thetags missing is common, so when computing the packagelocalization, it is likely that we cannot find a tag pair withthe same orientation. When we select tags with differentorientations, as mentioned above, the estimation errors ofthe perpendicular distances for the two tags are at differentlevels, which incurs the failure of localization. Accordingto Fig. 31, the probabilities of successfully localizing thetagged boxes are 97.5%, 90% and 77.5% with 2, 3 or 4

    boxes, respectively, so although using the 1D scanning isnot as robust as the 2D scanning, it is relatively reliable.One more important thing is to evaluate the alignment of acertain package stack, i.e., whether all package centers arealigned. As the 2D scanning splits the package stack withtwice orthogonal scanning, the accuracy of the alignmentevaluation depends on the perpendicular point accuracyalong the scanning direction. While the 1D scanning splitsstacks with the depth derived from the localization result,so the depth accuracy determines the alignment evaluationaccuracy for the 1D scanning. Fig. 32 shows the perpendic-ular point (l0) accuracy for the 2D scanning and the depthaccuracy for the 1D scanning. We find that the 2D scanningcan provide more accurate alignment evaluation accuracythan the 1D scanning, i.e., the median perpendicular pointerror is 1.77cm for the 2D scanning and the median deptherror is 8.3cm for the 1D scanning.

    5.3 Macro-BenchmarksAs the package stacking is based on the package orientation,we focus on the package orientation when comparing oursolution RF-3DScan and referred solution STPP.

    5.3.1 Accuracy with Different BoxesWe randomly choose a box from box1-3, and let the antennabe in front of the box about 1m to perform both sidesscanning of 0.5m 120 times. The results are shown in Fig. 33and Fig. 34. Fig. 33 shows the bottom face accuracy of thetwo solutions. It is found that our solution RF-3DScan canaccurately identify the bottom face with an average bottomface accuracy of 92.5%, while STPP has an average bottomface accuracy of 81.7%, RF-3DScan slightly outperformsSTPP by ×1.13. Fig. 34 plots the CDF of angle errors ofthe two solutions. It is found that RF-3DScan has the moreaccurate angle estimation than STPP. The median angle errorof STPP is about 3.58◦, while that of RF-3DScan is about2.52◦. RF-3DScan reduces 29.6% angle error compared toSTPP with different sizes of boxes.

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 14

    TABLE 2Bottom face accuracy with RF-3DScan and STPP

    True statusEstimated statuswith RF-3DScan

    Estimated statuswith STPP

    Normal Abnormal Normal AbnormalNormal 23 2 21 4

    Abnormal 2 48 3 47

    TABLE 3Precision and recall of mobile tagged packages

    Precision RecallRF-3DScan 92% 92%

    STPP 87.5% 84%

    5.3.2 Accuracy with Different DistancesWe select box1 and put it in front of the antenna movingplane with different distances between box1 and the antennaplane. We randomly vary the distance between box1 and theantenna plane within the range of [0.8m, 1.2m], and let theantenna perform both sides scanning of 0.5m 120 times. Theresults are shown in Fig. 33 and Fig. 35. For the bottomface accuracy in Fig. 33, it is found that STPP has an averagebottom face accuracy of 82.5%, whereas RF-3DScan achievesthe higher bottom face accuracy of 93.3%, outperformingSTPP by ×1.13. For the angle error in Fig. 35, it is foundthat the median angle error of STPP is 3.62◦. RF-3DScan canestimate the angle more accurately with the median error of2.13◦, reducing 41.1% angle error of STPP.

    5.3.3 Mobile Packages and Static AntennaTo simulate the conveyor situation, we let a static antennascan the mobile tagged packages to check the packagestatus, i.e. normal, rollover or upside down. For each status,we use the package to move both sides of 0.5m 25 times.The results are shown in TABLE 2 and 3. We combine theresults of ’rollover’ and ’upside down’ as ’abnormal’. It isfound that among all estimated ’normal’, the ratios of ’true’normal are 92% for RF-3DScan and 87.5% for STPP; whileamong all true ’normal’, the possibilities of the successfulidentification are 92% for RF-3DScan and 84% for STPP.

    Overall, our experimental results show that RF-3DScanscales better than STPP for different boxes and box-antennadistances. The average bottom face accuracies of RF-3DScanand STPP are about 92.5% and 82.5%, while the averageangle errors of them are about 4.08◦ and 5.05◦ separately,that is, RF-3DScan outperforms STPP by ×1.11 and ×1.12in terms of the bottom face accuracy and the angle error.

    6 CASE STUDYIn this section, we first give an example of the 3D recon-struction on tagged packages based on position indicatorsvia the 2D scanning. Then, we apply RF-3DScan to a morerealistic scenario, where tagged packages are distributedamong other packages in the non-line-of-sight situation.

    6.1 Case 1: 3D Reconstruction on Tagged Packagesbased on Position Indicators via 2D ScanningTo clearly show the idea of the reconstruction via 2D scan-ning in Section 4.4.2, we carry out this study for illustration.

    1st scanning 2nd scanning

    Fig. 36. Scenario of Case 1: Perform 2D scanning to determine packageplacement status

    1st scanning 2nd scanning

    (a) Result with 1st scanning

    1st scanning 2nd scanning

    (b) Result with 2nd scanning

    Fig. 37. Reconstruction results with 2D linear mobile scanning

    As shown in Fig. 36, we put four boxes of the same typeas box3 in front of the antenna and let the antenna performthe 2D scanning along two orthogonal directions. Box No.1and No.2 are in the normal state with angles of 0◦ and 30◦,while box No.3 is upside down with the angle of 0◦, and boxNo.4 is rollover with the angle of 60◦. For each scanning,the antenna is away from the boxes around 1m and scansboth sides of the boxes by at least 0.5m. With the first linearmobile scanning, we are capable of determining the packageorientation and estimating the package stacking under theassumption that all boxes face the antenna and no box isbehind another box. The result is shown in Fig. 37(a). Asbox No.1 is farther from the antenna, so it may be belowbox No.2, similarly, box No.3 may be below box No.4. Afterthat, we perform the second scanning along the orthogonaldirection, the result is shown in Fig. 37(b). If boxes arereviewed as in a same stack for the twice scanning, theseboxes are in a stack only with the up-down relationship,just like box No.1 and No.2. If not, these boxes are notin a stack, such as box No.3 and No.4. Finally, RF-3DScanwell reconstructs the placement of boxes with the correctorientation status and the angle error less than 2.5◦ via2D scanning. Different from the 2D scanning solution, thelocalization-based 1D scanning solution in Section 4.4.3 ismuch easier to perform. With only the first scanning of2D scanning, we can get the similar final result shown inFig. 37(b) via the 1D scanning solution. However, in termsof the accuracy, although the bottom face accuracy andthe angle error of 1D scanning are close to those of 2Dscanning, the alignment accuracy among packages is muchdifferent. The errors of the package alignment are 2.21cmfor 2D scanning and 7.96cm for 1D scanning, respectively,consistent with the results in Fig. 32.

    6.2 Case 2: Distributed Tagged Packages in Non-line-of-sight Situation

    It is not hard to check the placement of outside packages asthey can been seen by users directly, the main challenge isto monitor the placement of inside tagged packages withoutremoving covered packages as Fig. 38(a). Thus, we hide a

  • IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 15

    Inside StaticTagged Package

    CoveredPackages

    MobileAntenna

    StaticAntenna

    MobileTaggedPackage

    (a) Experimental setup

    1 2 3 4 5 6Bottom Face

    0

    0.2

    0.4

    0.6

    0.8

    1

    Acc

    ura

    cy

    RF-3DScan

    STPP

    (b) Bottom face accuracy

    Fig. 38. Case 2: Distributed hidden static tagged packages in non-line-of-sight situation with mobile antenna

    tagged package, i.e., box3, behind packages without tags, sothat we cannot see it unless removing the front packagesfrom the antenna side. We randomly select any face as thebottom face, the distance between box3 and the antenna isfrom 1m to 1.2m. For each bottom face, we use the antennato perform both sides scanning of 0.5m 15 times, so thatwe generate angle profiles to determine the coarse packageorientation, i.e., identifying the bottom face. Fig. 38(b) showsthat both RF-3DScan and STPP are able to identify thebottom face with a high accuracy in the non-line-of-sightsituation. RF-3DScan achieves an average accuracy of 93.3%,whereas STPP achieves an average accuracy of 87.8%.

    7 CONCLUSIONIn this paper, we present RF-3DScan, an RFID-based sys-tem to perform the 3D reconstruction on tagged packages.Through the 1D mobile scanning, RF-3DScan can determinethe package orientation for a single package and the coarse-grained package stacking for multiple packages, whilethrough the 2D mobile scanning, RF-3DScan can determinethe fine-grained package stacking for multiple packages.In the future, we will further improve our approach forpractical applications, and we wish our work can benefitthe logistic-related industry.

    ACKNOWLEDGMENTSThis work is supported in part by National Key Researchand Development Program of China under Grant No.2018YFB1004704; National Natural Science Foundation ofChina under Grant Nos. 61872174, 61832008, 61702257,61832005, 61872173, 61802169; Provincial Key Researchand Development Program of Jiangsu under Grant No.BE2017152; JiangSu Natural Science Foundation underGrant Nos. BK20170648, BK20180325; Hong Kong RGCResearch Impact Fund, R5034-18; Shenzhen Basic ResearchFunding Scheme under JCYJ20170818104222072. This workis partially supported by Collaborative Innovation Centerof Novel Software Technology and Industrialization. Lei Xieand Baoliu Ye are the co-corresponding authors.

    REFERENCES[1] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe,

    P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison et al.,“Kinectfusion: Real-time 3d reconstruction and interaction usinga moving depth camera,” in ACM Symposium on User InterfaceSoftware and Technology (UIST). ACM, 2011, pp. 559–568.

    [2] M. Firman, O. Mac Aodha, S. Julier, and G. J. Brostow, “Structuredprediction of unobserved voxels from a single depth image,” in

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2016, pp. 5431–5440.

    [3] P. Tanskanen, K. Kolev, L. Meier, F. Camposeco, O. Saurer, andM. Pollefeys, “Live metric 3d reconstruction on mobile phones,”in International Conference on Computer Vision (ICCV). IEEE, 2013,pp. 65–72.

    [4] C. Holenstein, R. Zlot, and M. Bosse, “Watertight surface recon-struction of caves from 3d laser data,” in IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS). IEEE, 2011,pp. 3830–3837.

    [5] Q. Lin, L. Yang, Y. Sun, T. Liu, X.-Y. Li, and Y. Liu, “Beyondone-dollar mouse: A battery-free device for 3d human-computerinteraction via rfid tags,” in International Conference on ComputerCommunications (INFOCOM). IEEE, 2015, pp. 1661–1669.

    [6] T. Wei and X. Zhang, “Gyro in the air: Tracking 3d orientation ofbatteryless internet-of-things,” in International Conference on MobileComputing and Networking (MobiCom). ACM, 2016, pp. 55–68.

    [7] Y. Zheng, Y. He, M. Jin, X. Zheng, and Y. Liu, “RED: rfid-basedeccentricity detection for high-speed rotating machinery,” in Inter-national Conference on Computer Communications (INFOCOM), 2018,pp. 1565–1573.

    [8] H. Jin, Z. Yang, S. Kumar, and J. I. Hong, “Towards wearableeveryday body-frame tracking using passive rfids,” ACM on Inter-active, Mobile, Wearable and Ubiquitous Technologies (IMWUT), vol. 1,no. 4, p. 145, 2018.

    [9] C. Wang, J. Liu, Y. Chen, L. Xie, H. B. Liu, and S. Lu, “Rf-kinect:A wearable rfid-based approach towards 3d body movementtracking,” ACM on Interactive, Mobile, Wearable and UbiquitousTechnologies (IMWUT), vol. 2, no. 1, p. 41, 2018.

    [10] H. Li, E. Brockmeyer, E. J. Carter, J. Fromm, S. E. Hudson, S. N.Patel, and A. Sample, “Paperid: A technique for drawing func-tional battery-free wireless interfaces on paper,” in CHI Conferenceon Human Factors in Computing Systems (CHI). ACM, 2016, pp.5885–5896.

    [11] J. Wang and D. Katabi, “Dude, where’s my card?: Rfid position-ing that works with multipath and non-line of sight,” in ACMSIGCOMM Computer Communication Review (SIGCOMM), vol. 43,no. 4. ACM, 2013, pp. 51–62.

    [12] W. Ruan, L. Yao, Q. Z. Sheng, N. J. Falkner, and X. Li, “Tagtrack:Device-free localization and tracking using passive rfid tags,” inInternational Conference on Mobile and Ubiquitous Systems: Comput-ing, Networking and Services (MOBIQUITOUS). ICST, 2014, pp.80–89.

    [13] L. Yang, Y. Chen, X.-Y. Li, C. Xiao, M. Li, and Y. Liu, “Tagoram:Real-time tracking of mobile rfid tags to high precision usingcots devices,” in International Conference on Mobile Computing andNetworking (MobiCom). ACM, 2014, pp. 237–248.

    [14] L. Shangguan and K. Jamieson, “The design and implementationof a mobile rfid tag sorting robot,” in International Conference onMobile Systems, Applications, and Services (MobiSys). ACM, 2016,pp. 31–42.

    [15] Y. Ma, N. Selby, and F. Adib, “Minding the billions: Ultra-wideband localization for deployed rfid tags,” in InternationalConference on Mobile Computing and Networking (MobiCom). ACM,2017, pp. 248–260.

    [16] H. Ding, J. Han, C. Qian, F. Xiao, G. Wang, N. Yang, W. Xi, andJ. Xiao, “Trio: Utilizing tag interference for refined localization ofpassive rfid,” in International Conference on Computer Communica-tions (INFOCOM). IEEE, 2018, pp. 828–836.

    [17] J. Wang, F. Adib, R. Knepper, D. Katabi, and D. Rus, “Rf-compass:Robot object manipulation using rfids,” in International Conferenceon Mobile Computing and Networking (MobiCom). ACM, 2013, pp.3–14.

    [18] J. Wang, D. Vasisht, and D. Katabi, “Rf-idraw: Virtual touchscreen in the air using rf signals,” in ACM SIGCOMM ComputerCommunication Review (SIGCOMM), vol. 44, no. 4. ACM, 2014,pp. 235–246.


Recommended