Automation & Robotics Research Institute (ARRI)The University of Texas at Arlington
F.L. LewisMoncrief-O’Donnell Endowed Chair
Head, Controls & Sensors Group
Talk available online at http://ARRI.uta.edu/acs
Wireless Sensor Networks Applications in UAV Helicopters and Intelligent Diagnosis
John Wiley, New York, 2006 John Wiley, New York, 2003
Cooperative Networks for Trust, Decision, & ControlWarfighter Information Network-Tactical (WIN-T) Concept of OperationsUS Army Signal Center
For warfighter:Extended sensory networkTrust verificationDecision fusion & assistanceControl over cooperating UAV & UGV
PDA
BSC(Base Station
Controller, Preprocessing)BST
WirelessSensor
Machine Monitoring
Medical Monitoring
Wireless SensorWireless
Data Collection Networks
Wireless(Wi-Fi 802.11 2.4GHz
BlueToothCellular Network, -
CDMA, GSM)
Printer
Wireland(Ethernet WLAN,
Optical)
Animal Monitoring
Vehicle Monitoring
Onlinemonitoring Server
transmitter
Any where, any time to access
Notebook Cellular Phone PC
Ship Monitoring
Wireless Sensor Networks
RovingHumanmonitor
Data Distribution Network
Management Center(Database large storage,
analysis)Data Acquisition
Network
ARRI Distributed Intelligence & Autonomy LabDIAL
UnattendedGroundSensors
SmallmobileSensor-Dan Popa
Testbed containing MICA2 network (circle), Cricket network (triangle), Sentry robots, Garcia Robots & ARRI-bots
Dr. Dan PopaMobile Robots
UGS-Xbow wireless sensor boards
• Temperature, ambient light, acoustic sensors, accelerometer,and magnetometer, (can get GPS)
• Each node has a microcontroller, programmable with a C-based operating system
• Cricket motes have ultrasound rangefinders
Environmental Monitoring & Secure Area Denial
Discrete Event Supervisory Control
Objective:Develop new DE control algorithms for decision-
making, supervision, & resource assignment
Apply to manufacturing workcell control, battlefield C&C systems, & internetworkedsystems
• Patent on Discrete Event Supervisory Controller • New DE Control Algorithms based on Matrices• Implemented on Intelligent Robotic Workcell• Implemented on Wireless Sensor Networks• Internet- Remote Site Control and Monitoring
USA/Mexico Internetworked Control
Man/Machine User Interface
TexasTexas
Intelligent Robot Workcell
Fast programming of multiple missionsReal-time event responseDynamic assignment of shared resources
Programmable MissionsMission Programming and Execution
Mission Programming for Distributed Networks`
R1
R2
R3
UGS1
UGS2
UGS3
UGS4
UGS5
Mission1-Task sequence
Mission 1 completedy1output
S2 takes measurementS2m1Task 11
R1 takes measurementR1m1Task 10
R1 deploys UGS2R1dS21Task 9
R2 takes measurementR2m1Task 8
R1 gores to UGS1R1gS11Task 7
R1 listens for interruptsR1lis1Task 6
R1 retrieves UGS2R1rS21Task 5
R2 goes to location AR2gA1Task 4
R1 goes to UGS2R1gS21Task 3
UGS5 takes measurementS5m1Task 2
UGS4 takes measurementS4m1Task 1
UGS1 launches chemical alertu1Input 1
Descriptionnotationmission1
Mission 2-Task sequence
Mission 2 completedy2output
R1 docks the chargerR1dC2Task 5
UGS3 takes measurementS3m2Task 4
R1 charges UGS3R1cS32Task 3
R1 goes to UGS3R1g S32Task 2
UGS1 takes measurementS1m2Task 1
UGS3 batteries are lowu2input
DescriptionnotationMission2
Fast Programming of Multiple Missions
DE Model State Equation:
DDucrcv uFuFrFvFx +++=
The Secret: multiply = AND & addition = OR
Tasks complete
Resources available
Targets / parts in
Command input
Task sequencing matrix – by Mission Planner
Resource assignment matrix – by Battlefield Leader
Fire next tasks
New Matrix Formulation for Supervisory Control
Discrete event controller
T asksco m p le ted v c
R u le-b ased rea l tim e con tro lle r
Cu curv uFuFrFvFx ⊗⊕⊗⊕⊗⊕⊗=
Job s ta rt lo g ic
R esource re lease lo g ic
W ireless Sensor
N etw o rk
. . .
u c
Se nsor ou tp ut u
R esourcere leased rc
S tart tasks v s
S tart reso urcere lease rs
O utp ut yM iss io n co m p le ted
P la nt co m m a nds P la nt s ta tus
D isp atch in g ru le s
C o ntro ller state m o nito ring lo g ic
xSv VS ⊗=
xSr rS ⊗=
xSy y ⊗= T ask co m p le te lo g ic
User interface:Definition of missionPlanningResource allocationPriority rules
U.S. Patent
Sensor readings
events
commands
Decision-making
Mission1-Task sequence
Mission 1 completedy1output
S2 takes measurementS2m1Task 11
R1 takes measurementR1m1Task 10
R1 deploys UGS2R1dS21Task 9
R2 takes measurementR2m1Task 8
R1 gores to UGS1R1gS11Task 7
R1 listens for interruptsR1lis1Task 6
R1 retrieves UGS2R1rS21Task 5
R2 goes to location AR2gA1Task 4
R1 goes to UGS2R1gS21Task 3
UGS5 takes measurementS5m1Task 2
UGS4 takes measurementS4m1Task 1
UGS1 launches chemical alertu1Input 1
Descriptionnotationmission1
Mission 2-Task sequence
Mission 2 completedy2output
R1 docks the chargerR1dC2Task 5
UGS3 takes measurementS3m2Task 4
R1 charges UGS3R1cS32Task 3
R1 goes to UGS3R1g S32Task 2
UGS1 takes measurementS1m2Task 1
UGS3 batteries are lowu2input
DescriptionnotationMission2
Fast Programming of Multiple Missions
Construct Task Sequencing Matrix Fv
Part A job 1Part A job 2Part A job 3
Part B job 1Part B job 2Part B job 3
Par
t A jo
b 1
Par
t B jo
b 1
Par
t A jo
b 2
Par
t B jo
b 2
Par
t A jo
b 3
Par
t B jo
b 3
Nextjobs
Prerequisitejobs
Used by Steward in ManufacturingTask Sequencing
Contains same informationas the Bill of Materials(BOM)
Mission Planner
Graphical User Interface
Construct Resource Requirements Matrix Fr
Used by Kusiak in ManufacturingResource Assignment
Contains informationabout available resources
Nextjobs
Prerequisiteresources
Part A job 1Part A job 2Part A job 3
Part B job 1Part B job 2Part B job 3
Con
veyo
r 1C
onve
yor 3
Fixt
ure
1
Rob
ot 1
-IBM
Rob
ot 2
-Pum
aR
obot
3-A
dept
Battlefield Commander
Graphical User Interface
⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜
⎝
⎛
=
100000000000100000000000100000000000110000000000010000000000010000000000011000000000001100000000000
19
18
17
16
15
14
13
12
11
1
xxxxxxxxx
Fv
⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜
⎝
⎛
=
000000000010000000000000000000000110000000000000000000111100000
19
18
17
16
15
14
13
12
11
1
xxxxxxxxx
Fr
⎟⎟⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜⎜⎜
⎝
⎛
=
100000100000100000100000100000
26
25
24
23
22
21
2
xxxxxx
Fv
⎟⎟⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜⎜⎜
⎝
⎛
=
000000000000010010000000000000000010000100
26
25
24
23
22
21
2
xxxxxx
Fr
High Level Controller
Dispatching rules
To Generate uc
RS232 RS232 RS232
Wireless Network with Internet connection
- -Rule Based Real Time Controller
ucStart tasks/jobs
Mission result
Resource release
Sensor output u
Task completed v
Resource released r
Medium Level Tasks ControllersRobot 1
Task 1 Task 1
Robot 2
Task 1
Wireless sensors
Task 1
Robot 3
RS232
Pioneer arm
Cybermotion robot
Cybermotion robot
Xbow sensors
Environment
Task1
PC
urv uFrFvFx ⊗⊕⊗⊕⊗=xSv VS ⊗=
xSy y ⊗=xSr rS ⊗=
Finite state machine for each agent
UC-TDMA MAC protocol
Supervisor control level A
gent control levelN
etwork control level
Agents
Mission 1 matrices
Mission 2 matrices
Events
Schematic Event Sequence for Mission Performance
LabVIEW Real-time Signaling & Processing
CBM Database and real time Monitoring
PDA access Failure Data from anytime and
anywhere
User Interface, Monitoring, & Decision AssistanceWireless Access over the Internet
Condition-Based MaintenanceMonitoring the System
Wired SensorsWireless Sensors
No long wiresRemote Monitoring
Predicting Failures in the SystemUse of Features from Empirical Data
Time Domain AnalysisSpectral Analysis
Use of various techniques for Classification
Fuzzy, Neural, etc.
Prasanna Ballal
WSN for CBMFeatures:• Scalable and energy efficient wireless sensor
network saves installation costs.
• Continuous and real-time collection of sensor data
• Low cost
• Portable hardware processor with diagnostic and prognostic tools
The PHM/CBM Cycle
MachineSensors
Pre-Processing
FeatureExtraction
FaultClassification
Predictionof Fault
EvolutionData
ScheduleRequired
Maintenance
Systems &Signal processing
Diagnostics Prognostics MaintenanceScheduling
Identify importantfeatures
Fault Mode Analysis
Machine legacy failure data
Available resourcesRULMission due dates
Required Background Studies
CBMPHM
SelectSensors!
Systems Approach to Intelligent Diagnostics & Prognostics
Dr. George Vachtsevanos http://icsl.gatech.edu/icsl
WSN and CBM
Prognostic tools
Turbine engine
Enhanced prognostic results
Base Station
Base Station
SN
SN
SNSN
Feed Back
Feed Back
DataBase
DataBase
DisplayDisplay
AnalysisAnalysis
The network can be made very reliable and energy efficient using UCTDMA (Tiwari, Ballal, Lewis 2007)
Wireless Sensors
Crossbow Mica2
Microstrain SG-Link accelerometer
McMiddleton Mote- built in-house at ARRI
Network Configuration Wizard
Useful for making minor changes to node parameters
Loads with Default Values for Parameters
Install and Configure the WS Network in 1 hour
DSP- Data to Information
Discrete Event - triggersAdvise, Decision Assistance, Alarm
LabVIEW GUIs Developed
Multiple Time Signal Display
Analysis and FFT
Decision-MakingDiagnosis & Prognosis Alarm Functions
Won U.S. Small Business Administration SBIR Tibbets Award
ARRI SBIR Program
Current SBIR DoE Small Business Innovation Research (SBIR) Contract, Phase I:
PIs F.L. Lewis and Dr. Weijen Lee
"Secure and Reliable Wireless Communication and Fault Diagnosis for Energy Control Systems,“
From Dr. Chiman Kwan, SignalPro, Inc., 9 mo. contract.
Prasanna Ballal
Electrical Faults Test-BedElectrical Fault Classification Test-Bed for Power Generators and
Motors • Electrical partial discharge (PD) or corona discharge (CD) can result in dielectric breakdown of the
electrical insulation and failure of switch-gear and motor windings.• Experience indicates that PD/CD occur years before failure, which leaves sufficient time to plan
corrective maintenance to avoid equipment failure.
Inductors to emulate winding fault Fault generator to emulate rotors
Hall effect sensorWSN
Dr. Weijen Lee
Mechanical Faults Test-BedMechanical Fault Monitoring for Power Generators and Motors Testbed
Motor
Fly Wheel
WSN
VibratingSensor
Dr. Weijen Lee
Fault Features
50 100 150 200 250 30010-4
10-2
100
102
Frequency(Hz)
Am
plitu
de
Wireless-Test data
FaultlessSolid-10mHSolid-20mHSolid-30mHRes10-10mHRes10-20mHRes10-30mH
30 40 50 60 70 80 90 100 110 120 130
10-2
10-1
100
PS
D a
mpl
itude
Freq no
Electrical Test-Bed Mechanical Test-Bed
Frequency Domain: Power Spectra
Faults in Rotating Machinery Generally Appear in the FFT SIDEBANDS
Fault FeaturesTime Domain: Mean, Kurtosis, Skewness
Image from www.joker-usa.com (distributor of the Joker 2 helicopter platform)
Communication Issues During Helo Aerobatic Flight
•The orientation of the helicopters changes continuously
•Antennas on the helicopters and on the ground station are not parallel
•Fading
Emanuel Stingu
Helicopter Control System Emanuel Stingu
Wireless communication systems: long-range & high-speed
GPS
Long Range14 mi
WiFiHigh speed
Helicopter 1
Helicopter 2
Helicopter 3
Ground Vehicle
Ground station(laptop computer)
Pilot 1(remote control)
Pilot 2(remote control)
Pilot 3(remote control) 900MHz long range, low speed
2.4GHz high speed 802.11n network
Radio links:
Wireless Communication TopologyThe helicopters have two radio communication interfaces:• Backup- Long-range, low speed: Maxstream XTend radio transceiver
900 MHz ISM band14 miles range with a 2.1dBi dipole antenna115,200 bps data ratepoint-to-point, point-to-multipoint, peer-to-peer and mesh topologies
• Control- Short-range, high-speed, low latency: Intel Wireless WiFi Link 4965AGN
2.4 GHz, 802.11n wireless network300 m rangeMIMO, diversity and three antennae support
GPSMagnetic compassAtm. pressure
Long rangeRadio Transceiver
Inertial unit
Ultrasoundrange sensor
Motor & rotor speedBatt. capacity
Battery
4 servomotors
Motor speed controller
Motor
Rotor speed transducer
Real Time & Computer
Module
802.11n antenna
Placement of the system componentsThe electronic components added to the system must not affectthe center of gravity – aerobatic maneuver capability is desired
GPS receiver and compass as far away as possible from the motor and the computer
The long range radio transceiver has 1W transmit power – has to be far from the various sensors
INU near the CG
E. Stingu
802.11n 2.4 GHz 5 dBi antennaTwo more to be installed
Antennas on the helicopter body
900 MHz 2.1 dBi dipole antenna(long-range comm.)
GPS helical antennainside the box
Antennas on the helicopter body
The dipole and the helical antennas do not require a ground plane, which makes it easy to use them on the helicopter body
Elevation
Azimuth
During aerobatic flight, the antenna attached to the helicopter can become perpendicular to the antenna on the other end of the link (base station, remote control or another helicopter).
Because the elevation pattern of the dipole is not uniform, depending of the orientation, it is possible for the received signal power to be as low as only 10% of the received power in normal conditions.
For small distances, the received signal is usually strong enough and a configuration with one antenna will be able to handle all orientations.
Issues regarding perpendicular TX and RX antennas
Almost all the signal power is lost when the antennas are perpendicular
Using 3 antennas to overcome the change in helicopter orientation
Extended Kalman Filter
Helicopter system model
Sensors on the helicopter: inertial unit, compass, GPS, pressure
Measurementsky~
)(tuInputs
States)(ˆ tx Decide which antenna is the closest
to the vertical
Antenna switching module
time
Switch the antennas only when the communication protocol allows it
(no RX / TX expected)
As the helicopter rotates, it will switch antennas such that the active antenna is the closest to the vertical from all three.
RSSI
RSSI
RSSI
Compare
Antenna with thebest reception
1
x1
xn-1
xn
1
y1
ym-1
ym
V W
Neural net
Extended Kalman Filter
Helicopter system model
Sensors on the helicopter: inertial unit, compass, GPS, pressure
Measurementsky~
)(tuInputs
States)(ˆ tx Get the helicopter
orientation
ϕ
θψ
NN Training algorithm
Using 3 receivers to determine how to choose the TX antennabased on the helicopter orientation
Receive phase during communications
Antenna selection signal
A neural network learns which antenna is better to be used for each orientation of the helicopter by analyzing the RSSI signal for each transceiver.
The 802.11n wireless network
• Intel 4965AGN mini PCI express wireless card• 802.11n standard – MIMO support already included• used for low-range, fast speed communication between helicoptersduring formation flight
Helix antenna for the GPS receiver
• More uniform radiation pattern allows the helicopter to tilt and pitch
• Problems appear for inverted flight
The GPS receiver
Communication fault diagnosis
Wireless card:
Communication statistics from Linux
Long-range transceiver:
RSSI, ambient power, transmission retries
Aerobatic flight
Hover
Automatic landing
Fault analysis
Communication errors Analyze Change behavior
