Tech United Eindhoven Team Description 2015

Cesar Lopez, Ferry Schoenmakers, Koen Meessen, Yanick Douven, Harrie van de Loo, Dennis Brui-jnen, Wouter Aangenent, Bob van Ninhuijs, Matthias Briegel, Patrick van Brakel, Frank Bot-

den, Robin Soetens, Tom Albers, Jan Romme, Stefan Heijmans, Camiel Beeren, Mar-jon van ’t Klooster, Remco Oudshoorn, Lotte de Koning, Rene van de Molengraft

Eindhoven University of Technology,Den Dolech 2, P.O. Box 513, 5600 MB Eindhoven, The Netherlands

www.techunited.nl, techunited@tue.nl

Abstract. In this paper we discuss the progress in mechanical, electrical and software de-sign of our middle-size league robots over the past year. Recent process in software includesintercepting a lob pass and improvement of 3D ball detection by combining omnivision out-put of different robots. For better localization the distance mapping and mapping center aredetermined during the game to compensate for tilt variations. To improve the accelerationcapabilities and agility of the three-wheeled robot an a-symmetric trajectory planner is de-veloped and implemented. Human-robot interaction is this year further explored by usinghand gestures to coach our robots. Finally a simulator link is developed to run a simula-tion with multiple computers which makes it possible to simulate matches of two full teamscompeting against each other.

Keywords: RoboCup Soccer, Middle-Size League, Multi-Agent Coordination, TrajectoryPlanning, Communication Interface, Human-Robot interaction, Calibration

1 Introduction

Tech United Eindhoven is the RoboCup team of Eindhoven University of Technology. Our teamconsists of PhD, MSc and BSc students, supplemented with academic staff members from differentdepartments. The team was founded in 2005 and originally only participating in the Middle-SizeLeague (MSL). Six years later service robot AMIGO was added to the team, which participatesin the RoboCup@Home league. Knowledge acquired during the design of our soccer robots provedto be a valuable resource in creating a service robot.

This paper describes our major scientific improvements over the past year. It is a part ofthe qualification package for the RoboCup 2015 World Championships in China and contains sixmain sections. First we introduce shortly our current robot platform. Next, Section 3 presentsour improved (3D) ball perception. This improvement is required to enable our next new feature,lob-passes described in Section 4. Section 5 discusses improvements in our localization algorithm.It presents an adaptation method to correct the calibration for tilt of the robot and tilt of themirror with respect to the turtle. An improved trajectory planner with a-symmetric accelerationand deceleration limits is described in Section 6. In Section 7 a new coaching method based onhand gestures is presented and the final section presents a simulator link, which makes it possibleto run one simulation using multiple computers.

2 Robot Platform

Our robots have been named TURTLEs (acronym for Tech United RoboCup Team: Limited Edi-tion). Currently we are employing the fifth redesign of these robots, built in 2009, together with agoalkeeper robot which was built one year later (Fig. 1). Development of these robots started in2005. During tournaments and demonstrations, this generation of soccer robots has proven to bevery robust. The schematic representation published in an earlier team description paper [5] stillcover the main outline of our robot design.

Changes regarding the robot platform, the 12V Maxon motors will this year be driven by ElmecViolin 15/60 amplifiers and two Lithium Polymer batteries [2]. A detailed list of hardware spec-ifications, along with CAD files of the base, upper-body, ball handling and shooting mechanism,has been published on a ROP wiki [1]. Our qualification page contains a detailed overview of themechanical and electrical hardware design and the software architecture. 1.

Fig. 1. Fifth generation TURTLE robots, with on the left the goalkeeper robot.

3 Ball perception

Last year a kinect sensor has been added to all robots to observe balls which are moving throughthe air. This sensor has proven its value when used by the keeper to detect the lob ball shot at ourgoal. However, there are some limitations with the kinect sensors when placed on the field players.For example, the kinect sensor has a limited view angle compared to for example omnivision.Therefore, the omnivision remains essential for ball detection. However, our current omninvisionalgorithm is heavily distorted by ball which are not on the ground. This section presents animproved ball detection method using omnivision for balls moving through free air. The ballsfound using this algorithm can be easily combined with the ball features detected by the kinect,which is further explained in Section 3.1.

1 2

z

x

Actual ball

Projected ball robot 2

Projected ball robot 1

Fig. 2. The intersection point of the vision lines of each robot represents the actual ball position.

The current omnivision algorithm projects the observed ball on the field to obtain its x and ycoordinate. This results in an offset if the ball is moving through the air. A possible solution toestimate this offset and to include the z coordinate of the ball is to use triangulation. To triangulatethe 3D position of the ball, at least two robots with vision on the ball are needed. The vision lines

1 http://techunited.nl/en/turtle-qualification

of the robots and their projected balls are known (Figure 2). If we know at least two vision lines,an intersect point of all lines can be calculated. The actual ball should be on the intersection pointof those lines. And therefore, the actual ball position (x, y, z) can be calculated which is neededto enable lob-passes, presented in the next section.

3.1 Ball model

A ball model is used to combine measurements from different sensors and to extract relevantinformation from the ball position measurements. A hypothesis-tree based sequential clusteringalgorithm [6] is adopted, with some adaptations to make it run in real-time and to deal withthe ball dynamics. An estimate of the current ball state is returned. Figure 3 shows an x, z plotof the output of the ball model based on simulated ball measurements. With the new omnivisionalgorithm, the input of the ball mode is improved and ball features from both kinect and omnivisioncan easily be combined to obtain an accurate 3D ball position.

−6 −5 −4 −3 −2 −1 0 1 2 3 4 5−0.2

0

0.2

0.4

0.6

0.8

x [m]

z[m

]

x

xv

Fig. 3. Ball state estimate for a simulated lob pass with noise. x is measured ball position, [x, v]is estimated ball state.

4 Lob passes

Another improvement of this year is the introduction of the lob pass. With the improved 3Dball tracking discussed in the previous section, the retrieved ball information is accurate enoughto intercept passes through free air. The pass-giver chooses a desired pass target, based on thecurrent game-play situation and several ball trajectory predictions. When the ball is shot, theoutput of the ball model is used to adjust the point of intercept for the shooting inaccuracy. Inthis section, the ball model and intercept strategy for a lob pass are discussed.

4.1 Intercept strategy

The receiving robot corrects for the shooting inaccuracy using a set of predicted bounce locations.The most favourable interception point is the current ball-tracking based prediction, derived frommeasured ball positions during the lob pass. The least favourable interception point is the initialestimate, as derived by the pass-giver before shooting. Figure 4 shows a flowchart-diagram of theintercept strategy. First, several validation conditions are evaluated. One of these conditions is anaccuracy constraint. Due to noise on the ball measurements, the ball-tracking based predictionscan be modeled as a bivariate distribution. The accuracy is derived from the sample covariance on abuffer of predicted locations. For all valid predictions, an intercept confidence rating is determinedbased on the estimated time of arrival and the magnitude and velocity of the incoming ball vectorat intercept.

Fig. 4. Basic outline intercept strategy.

Each prediction receives a hysteresis update. Finally, a target selector chooses a move-target,derived from the prediction with the highest hysteresis count. A move-target is different from anintercept target due to the ball-to-robot distance and an extra ’driving-forward’-distance. Thelatter is added to the intercept strategy to create a forward robot velocity at intercept. Thisincreases the catch rate for the current ball handling system, which is designed to catch flat passesrather than lob passes. The lob pass is tested in experiments. The robot trajectory is similar towhat is expected from simulation. To improve the catch rate further, the ball handling systemshould be extended for lob pass intercept.

5 Adaptation of localization parameters

For localization, line points in the camera image are being mapped onto an a priori known fieldgeometry. The left image in Figure 5 shows such a camera image. As explained in [3], this linepoints mapping is parameterized with 9 parameters:

• Distance mapping from pixels to meters (2 parameters)• Mapping center (2 parameters)• Robot center (2 parameters)• Robot pose (x,y,ϕ) (3 parameters)

These parameters are calibrated offline with a number of images at different locations using globaloptimization. By separating the mapping center and the robot center, small tilts of the cameraand/or mirror can be compensated for. If the mapping parameters are calibrated well, a mappedimage is obtained as shown in the middle image of Figure 5. If the parameters are not calibratedwell, the right image is obtained. A similar distortion is obtained when the robot is tilted, hence,this provides a possibility to compensate for tilts.

(a) Camera image with linedetection

(b) Camera image to fieldmapping with correctly cali-brated parameters

(c) Camera image to fieldmapping with shifted verticaloffset which is also observedwhen robot tilt occurs

Fig. 5. Camera images showing line detection.

Up to the year 2013, only the pose was being optimized during robot operation. The remainingparameters were fixed. Localization became difficult in case that tilt occurs during a game. To

compensate for tilt variations, from the year 2014, the distance mapping and mapping center arealso determined during a game. Hence, seven parameters are continuously being optimized foreach new image. This has shown to work well, tilts up to 2 degrees can be easily coped with. Thisis comparable to lifting one wheel by about 1 cm.

The computational load of the distance mapping had to be optimized to make this additionaloptimization possible in real-time. Images come at a rate of 50 Hz. Each image, 2 Nelder-Meadsimplex optimizations with maximally 300 cost function evaluations are run. Furthermore, for eachevaluation maximally 150 line points are being mapped. In total, about 5e6 line points have to bemapped each second. The remaining problem is chatter in both the ball and obstacles detectionif localization is lost. This happens occasionally e.g. due to objects that block the view. As thelocalization algorithm fails, the default mapping parameters are being used resulting in a transientin both the ball and the obstacle position predictions. To reduce this disturbance, slow adaptationof the mapping parameters is proposed. This will reduce ball/obstacle chatter when localizationis lost resulting in better ball intercepts in a dynamic environment with multiple obstacles.

6 A-symmetric trajectory planner

One of the major drawbacks of the three-wheeled robot of Tech United is its orientation dependentacceleration capability. This is caused by the angle between the wheels illustrated in Figure 6a.When the robot drives forwards or backwards, i.e. in the y-direction, wheel one (w1) and two(w2) are active while wheel three (w3) does not provide torque/force. If the robot drives in thex-direction or rotates, all three wheels are active and providing torque/force.

During a RoboCup game, the robot is mainly driving in the positive y-direction towards or withthe ball. During defensive actions or positioning to receive a pass, the robot will also driving in thex-direction. When the robot accelerates in the forward (positive y-) direction, the normal force ofthe front-wheels is reduced due to backwards tilt of the robot. In case that the robot decelerates,when driving in the positive y-direction, the normal force on the front-wheels is increased caused bythe inverse tilt. The friction force between the wheels and the ground is proportional to the normalforce of the robot to the ground surface. This friction between the wheels and robot determinesthe maximum acceleration of the robot without having slip. Hence, the maximum accelerationvalue is theoretically lower than the maximum deceleration value when driving forwards.

Ideally, the trajectory planner should take into account the wheel configuration, the centerof gravity and the grip of the omni-wheels to minimize the slip and maximize the accelerationand deceleration. An a-symmetric trajectory planner is proposed as a first step towards a moreeffective trajectory-planner. This trajectory-planner uses different limits for the acceleration andthe deceleration phase of a trajectory. The resulting trajectories are faster because the robot willdrive for a longer time at its maximum speed and the robot will be more agile because it candecelerate faster when its target changes while driving.

The trajectory-planner is implemented as a 2-DoF planner for x, y and a 1-DoF planner forφ. This means that the acceleration limit is treated as a 2-DoF constraint for x and y, i.e.,a2x + a2y ≤ a2max. An optional flag can be used to reduce the acceleration of φ such that theduration of the φ-trajectory is equal to the xy-trajectory. This reduces the slip probability in thesituation that the trajectory duration in xy is longer than the trajectory duration in φ. The outputof the planner is a second order position profile (discontinuous acceleration) which is used as aposition setpoint for the position controller of the robot.

Figure 6b illustrates an example of the effect of the a-symmetry in the trajectory. The robot ismoving towards position B at full speed (3 ms−1) and decides at position A to change its targetfrom position B to position C. The solid line illustrates the fastest trajectory to C if the maximumacceleration and deceleration are equal (1.5 ms−2). The black dashed line (- - -) is the resultingtrajectory if the deceleration is set to 150% of the acceleration. The red dotted line (-·-·-) is theresult of a deceleration set to 200% of the acceleration. It can be seen that the resulting trajectoryis much faster. The deceleration to acceleration ratio will be tuned and tested in several situationsto make the robot more agile during the game.

y

x

w3

w2w1

C B

A

(a) (b)

Fig. 6. (a) Techunited robot wheel configuration. The positive y-direction is the forward direction.(b) Example trajectory where the robot is moving towards position B at full speed (3 ms−1)and decides at position A to change its target from position B to position C. Blue solid line:deceleration = acceleration, black dashed line: deceleration = 1.5*acceleration, red dotted line:deceleration = 2*acceleration.

7 Human coaching

For the previous RoboCup MSL, the human-robot coaching was introduced making it possibleto instruct a robot with high-level instructions like ‘shoot more often’ or to change tactics, forexample how to take a free kick. To stimulate innovation, coaching by means of qr-codes is nolonger permitted at this years RoboCup. As a result, new ways of communication with the robotsis needed, leading to the introduction of hand gesture recognition by using the kinect cameras.

The principle behind the recognition algorithm is based on a dedicated geometric descriptor,where a convex hull and its convexity defects are used to detect the shape of the hand [4]. Usingthe openCV library one can easily extract this information out of the incoming images. While thisconcept has been proven to work almost flawless, some difficulties have been encountered duringthe implementation and experimentation of it, especially with the recognition rate in differentlighting conditions. As a result, a more robust concept will be developed, based on the seven Huinvariant moments on a binary image of the hand [7].

In order to achieve this, the incoming images first need to be filtered and converted to a binaryimage, to ensure a faster and more accurate detection of the hands. Hereto, two major steps areperformed, i.e. the filtering based on color and on depth. Filtering on depth implies losing all thepixels (set them to black or 0 in the binary image) that are too far away (over 5 meters) or tooclose by (less than 50 cm). The filtering on color implies that every pixel that has a color withina predefined range is set to 1 in the binary image, while all other pixels are set to 0. When donecorrectly, a binary image showing only the wanted hand to recognize will be the result. However,a well known problem with hand gesture recognition is the color of the hands, which is extremelydifficult to detect, especially in varying lighting conditions. To solve this, a glove with a distinctivecolor will be used, however an alternative could be working in the Y CbCr color space.

Once this binary image has been obtained, the Hu moments can be compared to the Humoments of known gestures stored in a database with the values resulted from many experiments.The goal of this new approach is to have at least five different hand gestures to be recognized witha 80% recognition rate, however testing should still confirm this.

8 Simulator link

The Tech United simulator enables RoboCup developers to simulate the behaviour of the TechUnited soccer robots without the need to run their program on actual robots. Therefore, the

development process gets expedited considerably. The simulator runs the same processes that areused on the actual robots with the exception that all modules interacting with the hardware havebeen replaced by modules that imitate the robots’ behaviour. Furthermore, the behaviour of theball is simulated using an advanced physical model.

By only simulating the behaviour of the low-level hardware, the simulator becomes highlyrepresentative of the behaviour of the robots on the field. One of the drawbacks of this approachis that it is computationally expensive because the software of several robots, which usually runson the separate embedded PCs of the robots, is now run on a single computer. As a consequence,the number of robots that can actually be simulated is limited. In order to increase this number,the so-called ’Simulator Link’ has been developed.

It is a communication interface, which makes it possible to connect two Tech United simulatorsthat run on different computers, and thereby makes it possible to simulate matches of two teamscompeting against each other. Every team is simulated on a separate computer, and the ’SimulatorLink’ communicates the data that needs to be exchanged, like the ball and robot positions, betweenthe computers. This is done by adding a communication module to the simulator, which handlesthe communication between computers and the different processes running on the same computer.An illustration of the communication architecture can be seen in Figure 7.

TR CSimulator(acti ve)

Server laptop

Turtl e 1

./motion sim

./vision sim

Turtl e 2

./motion sim

./vision sim...

......

TR C

Cl ient laptop

Turtl e 1

./motion sim

./vision sim

Turtl e 2

./motion sim

./vision sim...

......

Simulator(passi ve)

Fig. 7. Communication structure between a server and a client laptop both simulating a numberof robots.

To ensure a high enough data transfer rate, the communication between the two simulatorsis done using a UDP connection (strong line in the picture). The interprocess communicationis handled by mutex-protected shared memory structures (thin line). The data packages thatare exchanged between the simulators contain, besides the ball and robot positions, global robotsettings, like team color and activity status, the refbox commands, and shooting forces appliedby each robot. The input data is gathered from every simulated robot’s ’motion sim’ process. Onthe other computer, the robot positions are then translated to obstacle positions and are then fedto the ’vision sim’ process. The same happens with the ball positions. Refbox commands are sentfrom the simulator process to the ’motion sim’, where they are used as input to behavioural logic.

To prevent that the two simulators calculate different ball behaviours, it was chosen to onlyuse one of the simulators for these calculations, the ’Server’ simulator. The other simulator’s ballposition calculations are circumvented by using the communicated ball position as direct input.The second simulator process, the passive ’Client’, is only run to make use of its communicationinfrastructure so that it can distribute the data packages among the different simulated robots.Furthermore, Refbox commands can only be given by the ’Server’ simulator in order to avoidconflicting commands being issued.

Because the ’Simulator Link’ makes it possible to simulate two teams playing against eachother, it paves the way for high-level learning being applied to the logic that determines a team’s

strategy. One can imagine a scenario where two teams play against each other, while a learningalgorithm continuously adjusts the strategies. It is also possible to replace one team’s logic by amodel of an opponent’s team that was obtained using measurement data from previous matchesagainst this team. This way, the own strategy can be optimized autonomously using a learningalgorithm by simulating a high number of matches.

9 Conclusions

In our team description paper we have discussed concrete steps towards more accurate ball positionestimation which enables intercepting lob balls. Furthermore the localization software is improvedby compensating for small tilt of the camera during game-play. An a-symmetric trajectory planneris implemented to improve the acceleration capabilities and agility of the robots. Finally we havedeveloped hand gesture based human-coaching software and a simulator link which allows to runone simulation using multiple computers. Altogether we hope our progress contributes to an evenhigher level of dynamic an scientifically challenging robot soccer during RoboCup 2015 in China,while at the same time maintaining the attractiveness of our competition for a general audience.

References

1. Robotic open platform. http://www.roboticopenplatform.org/wiki/TURTLE.2. Specification lithium polymer batteries. http://kypom.com/cer_detail.asp?id=27, http://kypom.

com/cer_detail.asp?id=31.3. Dennis Bruijnen, Wouter Aangenent, Jeroen van Helvoort, and Rene van de Molengraft. From Vision

to Realtime Motion Control for the RoboCup Domain. In IEEE International Conference on ControlApplications, pages 545–550, Singapore, 2007.

4. Bruno Emile Jean-Francois Collumeau, Remy Leconge and Helene Laurent. Hand gesture recognitionusing a dedicated geometric descriptor. Image Processing Theory, Tools and Applications (IPTA),3rdInternational Conference on, 2012.

5. Tech United Eindhoven MSL. Tech united eindhoven team description 2014, 2014.6. H.J. Schubert. Sequential clustering with particle filters - Estimating the number of clusters from data.

Proc. 8th Intern. Conference on Information Fusion, 2005.7. Yanmin Yin Yun Liu and Shujun Zhang. Hand Gesture Recognition Based on HU Moments in Inter-

action of Virtual Reality. Intelligent Human-Machine Systems and Cybernetics (IHMSC), 4th Interna-tional Conference on, 2012.