1

ENS Platforms for Environmental Observatories

CENS Seminar August 26, 2005

Dustin McIntire

Bernie Yip

Kei Ho Hing

Prof. Bill Kaiser

2

Seminar Overview

• Environmental Observatories– What are they?– What makes them different?– Example Applications

• Embedded Networked Sensing (ENS) Application Classes• A New ENS Platform for Environmental Observatories

– Hardware Architecture– Software Architecture

• Spring 2005 EE209S Test Bed and Current Status– Test bed design and project description– Example EE209S project solution– Recent developments and improvements

• Future Work– A new design flow– Iterative development cycle description

• Collaborations

3

Environmental Observatories

What are they?• Multi-user / Multi-application systems

– Support more than one user and/or application possibly at the same time

• May be in remote or difficult to reach locations– e.g. James Reserve, Medea Creek, Merced River, White

Mountains, Costa Rica

• May have long dormancy periods with intermittent periods of high activity– could be user initiated or environmentally stimulated

• Contain high performance sensing equipment– High fidelity sensors often require large power draw for low

distortion and/or high gain– Relatively high instantaneous peak power consumption– Large data payloads possible (imagers may be MB to GB of data)

• Mote / Microserver architecture currently most popular for this application space

4

Environmental Observatories: Application Examples

• Micrometeorology– Cold Air Drainage investigations at James Reserve– Regular, triggered, and model-driven sampling of:

• Atmospheric temperature, humidity, others

• AMARSS Program at James Reserve– Comprehensive observation of forest soil

• subsurface and surface phenomena for understanding of plant growth– Regular, triggered, and model-driven sampling of:

• Micrometeorology, gas analysis, soil properties• Imaging (root growth)

• NAMOS (Profs. David Caron and Gaurav Sukhatme at USC)– Biological phenomena in aquatic environments deployed at JR.– Regular, triggered, and model-driven sampling of biologicals

• Relies on complex fluorometer instrument • Requires management of energy and data processing support

• Plant Phenology– High precision actuated imager tracking plant growth– Regular, triggered, or model-driven imaging

• NIMS– Increasing reliance on complex instruments for water quality, visible

imaging, thermal infrared imaging, and other sensors

5

ENS Applications in Observatories

• Always vigilant sensing with support for in-network processing– Favors mote architecture– Large processing demands may be offloaded into network

• Intermittent vigilance with scheduled processing– Current microserver solutions are an example– Require complete platform shutdown to reduce energy usage

• Always vigilant with scheduled demand for platform resources– Compatible with many observational problems in the environment– Work may be deferred until a later scheduled time

• Always vigilant with intermittent, on demand processing– Can be applied to many problems from environmental to security– Similar to human awareness levels– Processing demand according to

• Sensor event triggers• Model-driven scheduling

6

A New ENS Platform Approach

7

ENS Platform Design Requirements

• Long term low energy operation for extended lifetime• Support for high performance sensors and actuators

on demand• Utilize open source projects as much as possible

– operating systems– libraries– tools

• Provide broadband networking including:– user authentication – secure communications

8

ENS Platform Design Requirements

• Permit multiple users with simultaneous platform access

• Provide exclusive access to specific resources when required

• Provide accurate accounting and quota capability for constrained platform resources (memory, energy, bandwidth).

9

ENS Platform Approach

• Current microserver systems tend to operate only at the high performance and high power end of the spectrum

• From an operational efficiency perspective this is good!– Stargate vs. MICA2 Mote computation efficiency (next slide)

– 802.11x vs Zigbee communication efficiency

• But under utilized resources lead to lots of wasted energy• Need to utilize DPM (dynamic power management) to control

the energy usage– Can leverage lots of prior work done on processor centric DPM

• We require a real-time visibility into how energy in consumed at a fine grained level– Need feedback on how policy changes have affected the system

both at the micro and macro levels

10

Computing Efficiency vs. Power

• Low power does not always mean high efficiency• High performance CPU’s often offer high efficiency as well

11

Communication Efficiency vs. Power

• Again low power does not always mean high efficiency• High performance networking devices often offer higher efficiency

802.11g ZigbeeChipset Atheros 5006XS CC2420Output Power 16dbm 0dbmRx Sensitivity -78dbm@36Mbps -90dbm@250KbpsTx Power (Max Output) 1320mW 57.42mWRx Power 924mW 65.01mWTotal Power 2.24W 122.43mWEffective Throughput 20Mbps 125KbpsEfficiency (nJ/bit) 112 979

• Previous experiments with 802.11b and RFM TR1000 show the same results

12

Slauson + LEAP Hardware Architecture

Slauson(Sensoria PXA255 Platform)

LEAP (Low power Energy Aware Preprocesor)

SPII2C

Vsense

Shutdown

x5

+ -SY

STEM

_VIN

5.0_V

REG

3.3_V

REG

x2

Current SensedSupply Outputs

Sensor Inputs

USB Host

Flash

USB Host Controller

SDRAMPCMCIA1

PCMCIA2

Address/Data

Sensor Voltage

Ethernet

JTAGGPIO

SDCard/MMC

13

Slauson + LEAP Software Architecture

Slauson LEAPPXA255 MSP430

uC/OS

RedBoot Bootloader

Linux 2.6.12 OS

Busybox Cramfs

JFFS2 Flash File System

• Modified uC/OS for MSP430 to use msp-gcc compiler

• uC/OS is a small footprint RTOS with simple tasks, semaphores, etc.

• MSP code debuggable from remote PC (external JTAG) or Slauson PXA (GPIO JTAG) via msp-gdbproxy

• All open source GPL projects • ipkg capable file system updates

from local repo• Remote firmware upgradeable • Looking into OpenEmbedded or

Gentoo for flash file system building

14

EE209S Project Spring 2005

15

EE209S Test Bed Motivation

• Provide a general embedded systems teaching platform with little to no hardware specific knowledge necessary

• Allow students to be ‘idea’ constrained rather than ‘implementation’ constrained – Reduce much of the time spent debugging basic interfaces and

focus on the overall system architecture

– Provide many basic building blocks to construct a complex system from simple software components

• Allow greater latitude in algorithm exploration• Move from data acquisition/logging applications to event

detection and classification in a controlled environment• Provide research test bed for CENS and external groups

16

EE209S Test Bed Architecture

Internet

Internet

EE209S

Node 1

EE209S

Node 2

Linux PC

Test Bed System Components• Linux PC

– internet gateway to ENS network– cross compiler for nodes– user home directories and data

storage• EE209S nodes

– basic file system applications– ENS utility applications– user specific applications

• Events Generator– red/green light array– controlled from PC application

EE209S

Node N

EE209S

Node 3

…

17

EE209S Test Bed Example

ActuatedImageSensorNode

Field of ViewOf

Trigger Sensor

Event GeneratorServer

ViewingObstacle

• Green event detected and classified• Red event detected and ignored

18

EE209S Project Description

• Project goal is to detect and classify a finite number of known environmental events– results scored by accuracy and total system energy usage

• Events defined by simple state based contexts– A context is an issue rate and speed vector tuple

{slow,medium,fast}

• Student teams must correctly detect and classify all events with minimum energy usage

• Context duration or dwell time is varied• Students may change node behavior based upon current

context state and previous detection events– Many adaptive heuristics were used

– Some performed well at very low energy but missed transition events

– Others were extremely accurate but higher energy solutions

19

EE209S Node Architecture

Sony

Camera

Photo

Diode

Slauson

LEAP

Ethernet

ADC

802.11b

PCMCIA

PCMCIA

• Slauson + LEAP – provides local processing and power

duty cycling– provides micro-power vigilant state

• Sony SNC-RC30N Camera– 360 degree zoom/pan/tilt– power is duty cycled by LEAP

• 802.11b PCMCIA card– broadband communications channel– may be duty cycled by Slauson

• Silicon photo diode– micro-power detection sensor– used as a wakeup signal

20

EE209S Test Bed Software Architecture

Slauson LEAP

PXA255

Linux 2.6

MSP430

uC/OS

Power Mgt Scheduler

I2C Messaging

Tripwire Detect

MSP Client App

Energy AccumulationStudent Algorithms

Imaging Support Apps

Communications Apps

21

EE209S LEAP Preprocessor API

• Linux side msp-client application abstracts messaging API to MSP software– Removes the complex message framing and handshaking over I2C

bus– Handles bus collision and error recovery

• Simple command line interface that allows single or batched (file based) command sequences to facilitate scripting

• EmStar wrapper library would be a simple addition• Sample of msp-client commands:

– charge – read all charge accumulator values and optionally reset– sensor – read sensor ADC values and optionally do simple processing (min,max,ave.)– readtime – read the MSP current tick value to coordinate local time values– showpower – read the currently scheduled power management commands– showtrigger – read the voltage level setting for asynchronous resource wakeups– resetalarm – reset a previously triggered alarm– power – schedule a new power management command for future activation

(off,on,standby,resume) – hwreset – force a complete hardware reset of entire platform– trigger – set a new triggering voltage and/or channel for the alarm setting– settime – set a new clock time in the MSP– subscribe – subscribe a file name to be written with the time value of the next triggering event

22

EE209S Communications API

• 802.11 cards employ WPA encryption for secure physical layer• Node to node communications based on SSL layer for robust, secure

links– Pre-exchanged keys provide fast authentication– Key management mechanism will be discussed later

• OpenSSH based scp for file transfer and ssh for IPC• Created a set of basic command line utilities:

– ping_remote – detect the presence of one or more peers– copy_to_remote – send a local file to one or more peers– get_from_remote – retrieve a remote file onto the local file system– view_remote_directory – view the contents of a remote directory on one or

more peer nodes– remote_command – execute a shell command on one or more remote peers

23

EE209S Imager API

• JPEG image files retrieved from camera via FTP • Retrieved files are processed by local image processing

libraries for image centroid and intensity. Returns (x,y) vector for centroid location.

• Created a set of basic command line utilities:– camctrl – direct access to camera functions via command line

utility. Camera may be panned, tilted, zoomed, and images captured

– search_green – performs 360° rotation in 30° steps, images are green filtered and centroid located

– search_red - performs 360° rotation in 30° steps, images are red filtered and centroid located

24

EE209S Example Experiment Data

• Events: 10 {f,f} + 10 {m,m} + 10 {s,s}

25

EE209S Example Experiment Data

Power vs Time for Slauson5 (per component)

0

2

4

6

8

10

12

time 5034 10068 15099

Po

wer

(W

)r)

LEAP

Slauson

Camera

26

EE209S Example Experiment Data

Node Energy vs. Time

0

5000

10000

15000

20000

25000

30000

0 2000 4000 6000 8000 10000 12000 14000

Time (s)

Ener

gy

(J) slauson1

slauson2

slauson3

slauson4

slauson5

27

EE209S Algorithm Comparisons

1Performance (detection/classification accuracy %)

Energy

Transition Rate (context duration-1)

Algorithm A

Experiment 1

28

EE209S Lessons Learned

• Scheduling and coordination of system resources is not optional, it’s mandatory

– Each group with its own testbed is not practical for cost and space reasons– Unrestricted access for all groups led to numerous resource conflicts– Scheduling node usage via a simple web application (no enforcement) led to

accidental node contentions and low resource utilization– Need an automated resource scheduler and resource policing

• Manual log harvesting from nodes after experiments then merging is tedious– Need to provide a standard log format for simple offline post processing– Need to provide an automated means to terminate experiments and collect the log

records

• Iteration process is slow due to excessive manual intervention– Each iteration required code modifications to be pushed out to all nodes– Nodes had to be resyncronized and the experiments restarted by hand– Most of this process is redundant between iterations– Need to provide an automated means to remove old algorithms from the nodes,

push new algorithms to the nodes and restart all nodes synchronously

29

Current ENS Test Bed Progress

30

ENS Testbed Progress

• Added versatile resource scheduling and policing system to node test bed– Fine or course grained resources may be specified (e.g the serial

port on slauson1 or all nodes in test bed)– Users or groups must be on permitted access list for requested

resource– Users must request resource for specified time interval or be put on

a waiting list to be allowed access– Users must check-in at start of requested time interval to eliminate

no-shows. The waiting list is used if no check-in is received

• Added automated algorithm deployment and mechanism– At end of allocated time interval the previous experiment can be

terminated and processes killed. The previous user is optionally removed from nodes after saving the results directory

– The active user is deployed and his processes started from a specified startup file

– The active user may kill, deploy, and restart via single interface

31

Device Management Software Architecture

32

Device Management Software Architecture

System administration interface• Direct control of the system through access/permissions files• Run-time adding / removing devices, users, groups, and

schedules• Adjustable system resource allocation, e.g. reserved period

length, user reservation priorities

User interface• Sign-in in advance to reserve parts of the system for exclusive

access• Check-in when ready to use. Priority is given to the order of the

sign-in users, then waitlist users, and then drop-in users• User check-out prior to allocated time allows anyone to check-in

33

User Home Directory Structure

source

install

34

User Home Directory Structure

• modules - contains all the development system binaries– checkContent - check what is inside remote directory– checkProcess - check what processes are running by the current user– editCode - help .c and .sh file editing– getFiles/sendFiles - transfer file from/to slausons– getSysytem... - help maintain directory structure as shown in the graph– key-regen - regenerate ssh keys for slausons + put key in authorized_key– compile - cross compile source files for target architecture– push/pull - put/get all files in the 'install' directory to their corresponding slausons

• utilities – contains basic hardware access utilities for the target nodes described previously. Includes scripts to install user boot scripts and start user binaries.

• backup - contains tar files of a users home directory on each slauson node. They are created whenever the “pull” module is executed. The name of the tar files are followed by a date index.

• source - contains source code and/or scripts developed by a user for each node. The compile utility in the modules directory will compile the source code in this directory and place the binary files into the user’s bin directory for each node.

• install - contains any file that the user wants to be present on each of the remote slausons. It includes binaries located in “slausonx/bin” that is compiled using compile module from the source directory.

35

ENS Test Bed Near-Term Additions

• Imaging obstructions– Add known and unknown obstructions between the event generator

and the node imagers

• Color discrimination– Add light color discrimination utilities to node software

• Increase complexity for event descriptions– Current event context is a simple 9 state variable {ss,sm,sf,…ff} – May be expanded to include more complex events (e.g color,

direction, number of illuminated lights, etc)

• Unknown node ordering– Known fixed ordering may be replaced with unknown ordering– Would require a learning phase prior to running experiment

• Increased test bed dimensionality– Linear array can be changed to 2D or 3D version– Could add a mobile event or node

36

The Future – Bridging the Design Process Gap

37

The Design Process Gap

• Currently a system designer is unable to accurately account for assignment of energy and other resource usage to individual sensors, processes, and applications– Simple in early systems with small numbers of simple sensors

– Soon we will face ENS systems with multiple multimodal sensors

• Resource and Performance Profiler– Critical need for resource profiling tools (OS profiling tools starting

to appear e.g. LTT, KProbes)

– Enables an iterative design cycle and facilitates a continued improvement process

– Primary objective is energy efficiency

– Other objective functions also possible for other resource types (e.g. maximize off time for battery relaxation, minimize the peak to average power)

– Facilitates budgeting and scheduling of shared resources

38

Design Process Overview

ENS Algorithm Design

Coding and Scripting

Test Bed System Control

1 File distribution

2 System initialization

3 System termination

4 Log harvesting

Physical Devices - System Execution

0

1

2

3

4

5

6

7

8

9

10

0 5034 10068

Po

wer

No

de

5 (W

)

0

1

2

3

4

5

6

7

8

9

10

En

erg

y N

od

e 5

(kJ)

Event Generation in Matlab

Event Generator Control

Read test vectors and set the event generator lights (red/green patterns)

Visualization and Analysis

Algorithm Refinement

39

Design Process Elements

• Test bed system control provides functions to:1. Transfer files from/to remote nodes

2. Maintain system directory structure

3. System startup (e.g. time synchronize and execute from entry) and termination (e.g. killing user’s processes)

4. Log information harvesting

• Event generation in Matlab:1. User creates an abstract event type and saves to an event type file

2. Matlab assists in generating complex models for event sequence (e.g. HMM, stochastic process)

3. Index.txt contains list of event execution times. It is read and executed by the event generator control program.

40

Conclusions

• Environmental observatories are an increasingly important application of ENS systems

• Environmental observatories have special needs not addressed in traditional ENS design

• Specialized platforms and software are needed to address these shortcomings

• Increased system visibility during the algorithm design process is needed to improve results and speed deployment

• Iterative design process increases search of solution space and quantitative comparisons of algorithm alternatives

41

Collaborations

• ENS-box Concept from Winter 2004 CENS Retreat– Jeff Tseng and Richard Pon

– Design based largely on LEAP platform from EE209S

– PCB starting fabrication process

• Acoustic Localization– Lew Girod

– Slauson PXA255 platform with acoustic sampling and 802.11 WLAN via PCMCIA

• Linux 2.6 kernel and OpenEmbedded for Stargate– Martin and Naim

– Create an open source end-to-end distribution for microserver class platforms

42

Special Thanks

• Thank you to Bernie and Kei for their work on the project as well as their help collecting data and creating slides

43

Extra Slides

44

Environmental Observatories: Application Examples (Detailed)

• Micrometeorology– Example, Cold Air Drainage investigations at James Reserve– Regular, triggered, and model-driven sampling

• AMARSS Program at James Reserve– Comprehensive observation of forest soil

• subsurface and surface phenomena for understanding of plant growth– Regular, triggered, and model-driven sampling

• micrometeorology and gas analysis (air temperature, humidity, solar radiation) • soil properties (soil moisture, CO2, nitrate concentration )• Imaging (root growth – mini-rhizotron)• Atmospheric gas analysis

• NAMOS– Investigation of biological phenomena in aquatic environments deployed at JR. (Profs. David Caron and

Gaurav Sukhatme at USC)• Relies on complex fluorometer instrument • Requires management of energy and data processing support

• Plant Phenology– High precision actuated imager tracking plant growth– Image capture on regular, triggered, or model-driven schedule

• NIMS– Rapidly deployable arrays of coordinated fixed and mobile nodes– Increasing reliance on complex instruments for water quality, visible imaging, thermal infrared imaging, and

other sensors– Increasing importance of low energy to reduce reliance on energy storage for rapidly deployable systems

45

Model Driven Scheduling

• Light sensing– Model derived from past diurnal and seasonal behavior determine

when light sampling will occur

• Plant phenology– Weather conditions drive model that indicates when additional

imaging is required

46

Environmental Observatories

• New Architectures– New investigations drive requirements for capable instruments– Sensor device periperals are advancing– Node must support intelligent peripheral

• Node Applications– Node– Micropower Microserver

• Application Categories– Regular, deterministic schedule

• Preprocessor manages sensor assets• Processor applied only on demand (data storage or data compression and transport)

– Sensor Data Trigger• Multisensor trigger algorithm hosted on preprocessor • Processor applied on demand to adapt to changing environment context• Optional dedicated preprocessor

– Model driven• Multisensor model algorithm hosted on preprocessor • Processor applied on demand to adapt to changing environment context• Optional dedicated preprocessor

– EE209 Field exp