Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | madeline-jefferson |
View: | 216 times |
Download: | 0 times |
Bala Lakshminarayanan, Mike McCullough
Introduction
• US Army Night Vision & Electronic Sensors Directorate (NVESD)
• Network of acoustic and image sensors– Visible and IR
• Classification of civilian targets
Bala Lakshminarayanan, Mike McCullough
Motivation & Background
• Military targets ~98%– 3 levels of fusion process, acoustic and
seismic data
• Current civilian classification ~80%– Improve accuracy rating– Both civilian targets and personnel
classification– Image data
Bala Lakshminarayanan, Mike McCullough
Objectives
• Generation of data set of image and acoustic data– Development and fusion of moving target ATR
algorithms
• Establish methods to collect data and its “ground truth”
Bala Lakshminarayanan, Mike McCullough
SFTB Setup
SFTB Base Station
Node 3 Acoustic sensor
Node 1 Acoustic sensor Node 2 Acoustic sensor
Node 3 IR sensor
Node 3 Acoustic sensor
Node 1 IR sensor
Node 1 Acoustic sensor
Node 2 Visual sensor
Node 2 Acoustic sensor
Met station & GPS
Commands through DOS Scripts
GPS
GPS
GPS
Wireless Ethernet connection
Bala Lakshminarayanan, Mike McCullough
Sensors
• Indigo Alpha Thermal camera
• Pulnix TMC-7DSP Color camera
• Knowles BL-1994 Microphone
Bala Lakshminarayanan, Mike McCullough
General Test Conditions (1)
• 3 nodes each with hexagonal acoustical array of 7 microphones and imaging sensor– Nodes 1 & 3 have uncooled IR camera– Node 2 has visible color camera
• Nodes gather information simultaneously for 3 minutes
• Acoustic sensor turns on imaging sensors
• MUSIC algorithm for DOA estimation
Bala Lakshminarayanan, Mike McCullough
General Test Conditions (2)
• Targets moving on gravel & asphalt roads
• Fully exposed– Trees or other vehicles occasionally in the
way
• License plates on the targets are not readable
• Stationary sensors
• Daylight operation (9:30am to 3:30pm)
Bala Lakshminarayanan, Mike McCullough
General Test Conditions (3)
• Target motion– Constant speed– Stops midway
• Constant acceleration, deceleration• Stops for count of ten
• Each target traverses at 5, 10, 15, 20 mph• Start and stop outside FoV of nodes• Creation of different scenarios
Bala Lakshminarayanan, Mike McCullough
SFTB Operation (1)
• Attended mode– Short term data collection / Demo mode
– Collects 4 types of data
– Surveillance, directed, pan scanning
– SFTB_Base.exe, FullSim.exe
• Data collection mode– Pure data collection
– Collects 4 types of data
– Acquire.exe (Video and acoustic), MetEffects.exe
Bala Lakshminarayanan, Mike McCullough
SFTB Operation (2)
• Collected data– Acoustic .dat – Image .arf– Ground Truth .agt
• Filenames depends on sensor, node, scenario and targets
Bala Lakshminarayanan, Mike McCullough
Numbering System
• SSSN00000_0000
• SSS = camera name– IN1 = Indigo IR camera 1– IN2 = Indigo IR camera 2– PX1 = Polinex Visible camera 1– AC1 = Acoustic number 1
• N = node number (1-3)• 00000 = scenario number• _0000 = number indicating vehicle number (1-7)
– Can have multiple numbers multiple targets
Bala Lakshminarayanan, Mike McCullough
Number System Example
• in1200004_0003– In1 indigo thermal camera #1– 2 node 2– 00004 scenario number 4– _0003 target 3
• ac1300001_0056– ac1 acoustic array #1– 3 node 3– 00001 scenario number 1– _0056 both targets 5 and 6
Bala Lakshminarayanan, Mike McCullough
AGT Format
• Very similar to a class in a high level programming language
• Agt{
PrjSect {…}SenSect {
SenUpd {…}}TgtSec{
TgtUpd {…}}
}
Bala Lakshminarayanan, Mike McCullough
PrjSect
• Name = “sftb”
• Scenario = “00001_0001”
• Site = “nvis0306”
Bala Lakshminarayanan, Mike McCullough
SenSect
• Denotes the sensor section of the AGT
• Contains all of the SenUpd
• Name = “in11”– Denotes which sensor being used
Bala Lakshminarayanan, Mike McCullough
SenUpd
• LatLong
• Elevation
• Keyword “Frame #1”
• Keyword “AcousticAz: 0”
• Keyword “Nodeld: 1”
• Azimuth 75.9077301025
• Time 2003 160 16 22 34 587
• Fov 40.0 30.0
Bala Lakshminarayanan, Mike McCullough
TgtSect & TgtUpd
• TgtSect is the sector that contains all the target updates
• TgtUpd– Keyword “Frame: #1”– Time 2003 160 16 22 34 587– Tgt
• Range 48.0• TgtType “MAN”• Aspect 46.0• PixLoc 40 63
Bala Lakshminarayanan, Mike McCullough
ARF Info
• Automatic target recognition working group Raster Format
• Contains header, sub headers, footers, 1 or more images
• Supports multiple frames
• Supports 16 different image types in same file
Bala Lakshminarayanan, Mike McCullough
ARF Info
Rows, cols, version, type, # frames, offset…
Colormap, comments
Image file
Bala Lakshminarayanan, Mike McCullough
ImageJ
• NVL used a plugin for converting image from .arf to other formats– Image processing and analysis in java
• Formats – dicom, pgm, jpg, bmp, tiff, raw…
• Operations – FFT, convolution, fractal box count, morphological…
Bala Lakshminarayanan, Mike McCullough
Acoustic Data
• Raw data in acoustic .dat file
• Contains header information for system time (similar to AGT), node #, longitude/latitude (0’s), and bearing (0’s)
• make_wave.py to convert from .dat to .wav– Changes to specify output file needed within
the python script to make it work properly– Script drops .dat extension and adds .wav
Bala Lakshminarayanan, Mike McCullough
Targets
• Vehicle (?) (target 5)– Names obtained from AGT files that would eventually
contain a TgtType indicating the target
• Toyota 4Runner (target 6)
• Stake body light truck (target 7)– Dave Rankins
Bala Lakshminarayanan, Mike McCullough
Example Data
• Convert arf files into raw using ImageJ
• Modify raw image into PGM– Switch endianness
• Apply image processing techniques to the image– Very hard to distinguish objects due to having a
dark image
Bala Lakshminarayanan, Mike McCullough
Example Sound Clips
• Normalized data using shareware program
• Target 1 – Node 1– Node 2– Node 3
• Target 6– Node 1– Node 2– Node 3
Bala Lakshminarayanan, Mike McCullough
Example Images (1)
• Sensor placement• FoV of a sensor covers atleast half of total FoV
Bala Lakshminarayanan, Mike McCullough
Future Work
• MatLab acoustical analysis
• Segmentation & shape analysis
• Feature selection & extraction
• Fusion
• Target Recognition algorithms