San Diego State University
College of Engineering
A Web-Based Mobile Robotic System for Control and Sensor Fusion Studies
Christopher Paolini
1, Gerold Huber
2, Quentin Collier
3 and Gordon K. Lee
1
San Diego State University
College of Engineering
Outline of Presentation:
The Mobile Robotic System Overview
The ANFIS Algorithm
Sensor Integration
Graphics User-Interface
Results
Conclusions and Future Work
San Diego State University
College of Engineering
Goal:
Develop a mobile robotic testbed to investigate several control algorithms and sensor fusion techniques
Approach: Use web-based video streaming and embedded control architecture for flexibility and robustness
San Diego State University
College of Engineering
iRobot Create® Platform
• Single Board Computer (SBC)
• Linux Voyage
• Unibrain Fire-i™ Digital Camera
• Linksys Wireless-G PC adapter card
• iRobot Create® platform
• Single board computer (SBC)
• Voyage Linux (Debian)
• Unibrain Fire-i™ digital camera (IEEE 1394)
• Proxima 802.11g PCMCIA adapter card with external 5db gain antenna
San Diego State University
College of Engineering
iRobot Create® with Sensor Arrays
• iRobot Create® based robot designed with several sensors
• Streaming video
• Web teleoperation
• ANFIS automation
Experimental iRobot developed at San Diego State University
Unibrain Fire-i™ Digital Camera
802.11g PCMCIA
Transceiver
360° Ultrasonic Sensor Array
Arduino Mega MCU
2.5dBi gain indoor omni-
directional antenna
Thermal Sensor
Migrus C787 DCF-P single board
computer with a 1.2GHz Eden ULV
Processor
IR Sensor
9DOF Inertial Measurement Unit
San Diego State University
College of Engineering
Sensors
3-Axis Rate Gyroscope 3-Axis Accelerometer 3-Axis Magnetometer
Inertial Measurement Unit Controller
San Diego State University
College of Engineering
Ultrasonic sensor array using an Arduino Mega 2560 microcontroller
San Diego State University
College of Engineering
MaxBotix LV-EZ1 Ultrasonic Sensor
An array of 10 MaxBotix LV-EZ1 sensors suspended
on two circular plates.
Each MaxBotix sensor provides a 36 degree FOV.
Ultrasonic Sensor Array
San Diego State University
College of Engineering
Thermal Sensor
How the iRobot Adjusts Its Heading
The iRobot changes its azimuth by sending a 16 bit signed
value in the range [-2000, 2000] mm that defines a
turning radius
The turning radius is a ray from the center of the turning
circle to the center of the robot
r > 0 robot turns left
r < 0 robot turns right
Special cases: r = 32768 or 32767 (0x8000 or 0x7FFF)
causes robot to move straight
r = 0xFFFF robot turns in place clockwise
r = 0x0001 robot turns in place counter-clockwise
r2r1
Large radius small curvature
Small radius large curvature
How the Web GUI Determines the Turning Radius
(x,y)
1tanx
y
23.8 2142.79r
In quadrant I and IV for >5
In quadrant II and III for <-5
23.8 2142.79r III
III IV
Assume r varies linearly with
• Robot has an onboard 9DOF IMU
• Incorporates four sensors: LY530ALH single-axis gyro, LPR530AL dual-axis gyro, ADXL345 triple-axis accelerometer, and a HMC5843 triple-axis magnetometer
• Gives nine degrees of inertial measurement
• LY530ALH: STMicroelectronics ±300 °/s analog yaw-rate gyroscope
• HMC5843: Honeywell HMC5843, a 3-axis digital magnetometer outputs Euler X,Y,Z orientation vectors and roll () and pitch () angles (tilt sensor) with 12-bit ADC at 10 Hz
• MCU computes azimuth or “yaw” with an accuracy of 1-2 º
Inertial Measurement Unit (IMU)
1
cos sin sin cos sin
cos sin
tan
h
h
h
h
x x y z
y y z
yAz
x
• We want the absolute bearing defined by the remote user with the GUI to equal the absolute bearing reported by the IMU
• How to compensate for unknown system dynamics?
• Can define a neural network to model robot (plant) dynamics and use training data to tune network parameters
• Define a set of 4-tuple training data:
which are the ith desired (from GUI) azimuth, azimuth rate, actual (from IMU) azimuth and azimuth rate, respectfully.
How to Model Robot Dynamics while Turning?
1 1 1 1
2 2 2 2d d a a
d d a a
n n n nd d a a
id
id
ia
ia
From the LY530ALH yaw rate
sensor
From the HMC5843 digital
magnetometer
San Diego State University
College of Engineering
ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM
AJAX based Web interface for telerobotic control of the iRobot Create Encoding bit rate as a function of fps
Virtual Force Field Approach for Obstacle Avoidance
The Certainty Grid
• 2-Dimensional array of cells
• Each cell contains a Certainty Value (CV)
• CV indicates the measure of confidence that an object exist within a cell
• Instantaneous map for obstacle representation
• dx = dy = 15 cm
• 21 by 21 square cells represent the Certainty Grid
Fron
t
Method for Updating Certainty Values
Each sensor corresponds to a particular angle Ө, based on
its position on the sensor assembly
At a given time, a sensor returns a distance d
Eq. 1 and 2 transform (d, Ө) → (x’,y’)
(1)
(2)
Obstacle Avoidance for Path Planning Task
Port S
ide
Front
San Diego State University
College of Engineering
Cells Located on the Acoustic Axis
San Diego State University
College of Engineering
(a) Histogram Grid ; (b) Snapshot of Video Camera
San Diego State University
College of Engineering
A1
A2
B1
B2
w 1
w 2
f
x y
x y
x
y
layer 1
layer 2 layer 3
layer 4
layer 5
w 2
w1 w f1 1
w f2 2
If x is Ai and y is Bj, then iiii ryqxpf
i
n
1iifwf
r
The ANFIS Architecture
San Diego State University
College of Engineering
Forward pass: consequent parameters
X i1 Xi Si1 ai1(b i1T ai1
T Xi )
Si1 Si Sia i1ai1
T Si
1 ai1T Siai1
, i 0, 1, .. ., P 1
E Ei (ydi y i)2 ei
2
i1
P
i1
P
i1
P
Off-line Training
Backwards pass: premise parameters
San Diego State University
College of Engineering
Forward pass: consequent parameters
On-line Learning
Backwards pass: premise parameters
San Diego State University
College of Engineering
San Diego State University
College of Engineering
Simulation Results
System Integration through an Arduino MCU
ANFIS controller is being implemented on an Arduino Mega that uses the magnetometer and accelerometer output for path tracking.
Initial simulation studies have been conducted using a MISO controller to test this proof of concept.
Then performed actual experimentation using one and two inputs to the ANFIS controller
• Multiple sensors: thermal (person/fixture differentiation), ultrasonic (collision avoidance and path planning), IR (automatic docking), video (telerobotic control), and magnetometer and accelerometer (orientation and position) have been tested and integrated into the overall system architecture.
Initial Web Browser Interface
San Diego State University
College of Engineering
San Diego State University
College of Engineering
Experimental Results
Bearing Scenario
San Diego State University
College of Engineering
Bearing Scenario
San Diego State University
College of Engineering
Constant Turn
San Diego State University
College of Engineering
Following Scenario
San Diego State University
College of Engineering
Learning Scenario
San Diego State University
College of Engineering
ANFIS Responding Time
Response Time
Inputs Membership-functions ANFIS Computation Time
1 2 3-4 16-18
3 7-8 20-23
4 9-11 23-26
5 13-15 26-29
6 17-19 30-32
7 22-23 36-38
2 2 12-14 26-28
3 30-31 44-46
4 80-82 94-96
San Diego State University
College of Engineering
Conclusions and Future Work
•A MIMO ANFIS controller has been designed and tested through simulation and experimental studies
•The desired controller can adaptively adjusting to system variations through supervised and un-supervised learning.
•Future tasks include extending the MIMO design to a multiple inputs, two output structure and evaluate the performance of this MIMO implementation using Player/Stage and experimentation.
•We will add on-line learning functionality to our embedded MIMO ANFIS that will effectively tune the parameters computed from off-line training data.
San Diego State University
College of Engineering
San Diego State University
College of Engineering