See3PO - A Visually Capable Path Finding RobotSee3PO

Frank Marino, Nick Wang, Jacky Yu, Hao Wu and Debarati Basu

Department of Computer ScienceUniversity of Massachusetts at Boston

Tyler Garaas and Srikumar Ramalingam Mitsubishi Electric Research Laboratories (MERL)

See3PO - A Visually Capable Path Finding Robot

• See3PO– Our robot, equipped with two cameras, will use

vision to navigate its environment

• Host– An application that runs on a Host PC, allowing the

operator to set up and direct the robot.

Overview

See3PO - A Visually Capable Path Finding Robot

GUILocal Brain

Floor Plan Processing

Locator

Host

PathFinder

Host PC

Floor PlanPositionDestination

Path

FP ImageFloor Plan

FP Image Destination

Position

Floor PlanRobot Images

Robot Images

Commands

Robot

Host

See3PO - A Visually Capable Path Finding RobotHost – An overview

The Host class manages all Host-Side processes1.The User selects a Floor Plan Image from the GUI2.Host sends this image to FloorPlanProcessing, which returns a FloorPlan.3.This FloorPlan is sent to PathFinder. It is also sent back to GUI for display.

See3PO - A Visually Capable Path Finding RobotHost – An overview

4. Either the User (GUI) or the Locator sets the robot’s current Position, then the User (GUI) selects a Destination.• If using the Locator, the Host requests images

from LocalBrain, and sends these to the Locator.5. These points are sent to the PathFinder, which

returns a Path of (X,Y) coordinates.6. Host converts this Path to distinct

Left/Right/Duration MoveComand objects, and sends them to the LocalBrain, one at a time.

See3PO - A Visually Capable Path Finding Robot

• FloorTile – represents one square foot on a floor.– Walkability - We can hold a range of values, but currently, we only use:

• Floor = 1• Wall = 0

– End/Start – booleans showing whether this is this tile is the start or endpoint of the graph.

– Neighbors – the four neighboring FloorTiles• FloorPlan

– A graph of Floor Tiles – Each tile is connected to each of its four neighbors if that neighbor’s

walkability is not zero.– Start/End – pointers to the start and end tiles

Floor Plan Processing

See3PO - A Visually Capable Path Finding Robot

• The getPath() method takes a start point and a destination point and returns the shortest path between these. – Currently uses Dijkstra’s algorithm for shortest path

in order to account for varying “Walkability” of tiles.– Returns a list of FloorTiles, which contain XY

coordinates.

Pathfinder

See3PO - A Visually Capable Path Finding Robot

Science Building – 3F (image)Graph

wallwalkable

s

t

s

t

Floor Plan

Processing

Generate

Move Command

Position FinderConfirm Position

s

t

See3PO - A Visually Capable Path Finding RobotGUI

See3PO - A Visually Capable Path Finding RobotGUI

Live PanelDisplays a live view from the Robot

Status ScreenDisplays detailed messages from the Host

Floor PanelThe Main Interface:•Displays The Floor Plan

•Allows the user to set:•Scale•Current Position•Destination

•Displays the best Path

•During execution, updates the Path and Position.Connection Status Instructions

See3PO - A Visually Capable Path Finding RobotLocator

Returns the current Position of the robot: Location(X,Y)

Facing (0 -359)

This position is used by PathFinder to create a Path, and to monitor and correct the robot’s progress while moving.

The only external inputs required are:A Floor Plan ImageRobot Images from the robot’s two cameras

                                                         contd..

See3PO - A Visually Capable Path Finding RobotLocator

Advantages:

Precise Works Indoors Self-correcting Recognizes changes 

  open/closed door   obstacles

See3PO - A Visually Capable Path Finding RobotTesting

Unit tests – are already in place

System tests – test plans are being developed

System testing had been done partially

See3PO - A Visually Capable Path Finding Robot

Robot

Main parts of the robot• Local Brain

– Receive commands from Host

– Send commands to ATOM and Servos

– Send image back to Host

• ATOM

– Control wheels

• Servos

– Control cameras

• Cameras

• Wheels

See3PO - A Visually Capable Path Finding Robot

Cameras

Local Brain

ServosAtom

Wheels

The Anatomy of a See3PO

See3PO - A Visually Capable Path Finding Robot

Communication Protocol

• Command sent from Host to local brain (TCP)

• Command sent from local brain to ATOM (RS-232)

Byte 0 1 2 3 4 5 6 7 8 9

0x01 0x10 0x11 LWS_H LWS_L RWS_H RWS_L TM_H TM_L 0xEF

Byte 0 1 2 3 4 5 6

0x01 0x10 0x11 LWS_H LWS_L RWS_H RWS_L

See3PO - A Visually Capable Path Finding Robot

What we need to know

• pulsout pin12, 8 – duration 8*0.5µs =4µs

• The longer the pulse, the faster the speed.

Syntaxpulsout pin, time pin: the I/O pin to use. time: the duration of the pulse in 0.5 microseconds increments.

time

See3PO - A Visually Capable Path Finding Robot

High Packet Loss Rate-ATOM

• Original Codemain: “send speed to wheel every 20ms

for i = 1 to 5 pulsout p12, rightSpeed pulsout p13, leftSpeed pause 20 next “read packet through COM port SERIN S_IN,i9600,… …goto main

Why????

If the packet come in during the loop…

See3PO - A Visually Capable Path Finding Robot

Our Solution

• New Code ONINTERRUPT TIMERAINT,drive ENABLE TIMERAINT main: “read packet in from COM port SERIN S_IN,i9600,… … goto main

drive: pulsout p12, rightSpeed pulsout p13, leftSpeed resume

What did we implement?

Interrupted by a timer every 20ms

See3PO - A Visually Capable Path Finding Robot

Robot cannot move straight

• Original CodeleftSpeed = 3000 rightSpeed = 3000main: gosub drive

gosub receive_commandgoto maindrive:

if initialized = 1 then for i = 1 to 5 pulsout p12, rightSpeed

pulsout p13, leftSpeedpause 20next

endif return

See3PO - A Visually Capable Path Finding Robot

How do we solve that

• Send different speed to the wheels– Take the move forwards

for example

– leftspeed=3179

rightspeed=3150

Note: these data comes from experimental testing

See3PO - A Visually Capable Path Finding Robot

Local Brain UI

• Debug mode – show commands

• Movement control – forward

– left

– stop

– ..

• Setting left and right speed as well as driving time

See3PO - A Visually Capable Path Finding Robot

What can we expect for version 1.0?

• A working servo system• The robot can send pictures and communicate with the

host from anywhere within wireless range.• Based on these images, the Host can return a location. • After giving a destination, the host can find the shortest

path between the robot and any destination.• Using this information, the robot will successfully

navigate to the destination.

See3PO - A Visually Capable Path Finding Robot

Possible extras

• Allow the user to change destination while the robot is moving.

• Allow optional manual control over the robot.• Account for doors and obstacles.• Use new wireless system (we use the wireless

card, so it can just cover about 10 square meters)– Use local internet (like the UMass wireless) to make

the robot move in the whole campus– Use other types of connections, like a tethered

iPhone, extending the range further.

See3PO - A Visually Capable Path Finding Robot

Questions?

• http://code.google.com/p/see3po/

• Please feel free to ask questions.