Gigapixel resolution time-lapse imaging for phenological monitoring of every plant in a landscape.

http://bit.ly/CBR-1

Tim Brown, Postdoctoral Fellow, Borevitz Lab, Australia National Universitywww.borevitzlab.anu.edu.au

To address the environmental and management challenges of the 21st century we need a substantially better understanding of how ecosystems work.

– Exponentially more data– Better models– Long time-series data sets

2

Ecology – Where’s my PCR?

Understanding high-order complexity is hard!1. The environment is complex and continuous but we typically only

measure limited data over limited snapshots in time

2. Difficult and expensive to track change on the ground at high spatial and temporal resolution

3. Long term data sets are rare, particularly of images– Satellite data is good but not high enough resolution for many applications

4. Hard to maintain field-based research over ecologically meaningful time periods with high enough sample size

To understand ecosystems we need to be tracking “everything” in the environment at high time-rates for long periods of time.

• What if we could watch every plant in our field sites from our desk?– Map out population genetics, biotic/abiotic data on

the landscape– Slide back in time and watch any interaction for as

long as there have been sensors– Students start new research projects beginning with

all the data previously collected at a site

4

The technology is here – We need to dream big!

Collaboration between

Borevitz Lab (U. Chicago, USA) and TimeScience (my company)

The Challenge:• Build a solar powered, weatherproof gigapixel camera that can record daily

phenology from every plant in a field area.

Gigavision: gigapixel timelapse camera

(Single 15MP image)

Area: ~7ha

Area: ~1m2

The Gigapan and Gigavision systems allow you to capture hundreds or thousands of zoomed-in images in a panorama.

Images are then “Stitched” into a seamless panorama.

The super-high resolution of the final panorama lets you monitor huge landscape areas in great detail.

Gigapixel Imaging – How it works

• ~1.5 billion pixels / panorama

• Avg. resolution of ~1 pixel / cm over 7 hectares

– (~600 million times the resolution of MODIS)

• Open-source - Built with off-the-shelf components

• Cellular (3G) or 802.11g wireless access (160MP “thumbnails”)

• Automated capture up to 1 image / hr

• Solar powered (<15w power consumption)

• ~$30,000 -> could be more like $10-15,000

• “Light-phenotyping” of >500 plants for ~$60/plant

The Gigavision Camera – Specifications

For full specs, see Brown et al. 2012.(Google: “gigavision chapter”

Camera Field of View (FOV)

Dataset statistics• Oct 2009 – Oct 2011

– 2 Growing seasons (April – Oct)• 1-4 panoramas / day (~154 images/panorama)• >184,000 individual jpg images captured• Processed data = 70 million 200x200pixel images• 6TB of space• 417 usable noon panoramas

11

12

Growing seasons

Image Visualization and Data Collection

http://www.gigavision.org

14

http://bit.ly/GVDemoMovie2012

Species

• 513 individual plants identified• 8 prominent species (non grasses)• Species:

– Hoary Puccoon = 344– Unidentified (yet) = 52– Cottonwood = 47– Black Oak = 36– Sand Cherry = 18– Juniper = 9– Wormwood = 3– Pitcher’s Thistle (Endangered) = 2– Marsh Marigold = 2

15

Limiting factor is increasingly software• For example:

– Axis Q-6035e• $4,000 USD• Can run on-board software• ~2 gigapixel image in < 10 min at any focal length• Temp range: -40 to +50C• 50W

16

GigaPan – Low Cost Gigapixel Imaging

GigaPan (non-timelapse)• $350-$1,000• Works with any camera• Great for documentation and low time-resolution monitoring (e.g. monthly, annual)

Example: • Alta Ski Area Bark Beetle Project

– Maura Olivos, Alta Environmental Center– Annual gigapixel survey images– Identify beetle infested trees for removal– Online panoramas: http://gigapan.com/galleries/5582/gigapans

• More examples of gigapans here: • http://gigapan.com/profiles/TimeScience

• GigaPan hardware: http://www.gigapan.com/

Alta Bark Beetle Project – Initial survey path for potential panorama locations

Data collected with EveryTrail smartphone app (http://www.everytrail.com/ )

Panoramas: http://gigapan.org/galleries/6787/gigapans

(1) Collins Weather

(2) Baldy Shoulder

(3) Road Shot

(4) GrizzlyBrowse all panoramas online here: http://gigapan.org/galleries/5582/gigapans

21

22

23

Project summary statistics

Site Name Approximate Area (Acres)

Image Resolution (Gigapixels)

Average Pixel Resolution

(Pixels per square inch)

Collins Weather Station 697 4.07 0.93

Baldy Shoulder 770 8.84 1.83

Road Shot 211 3.26 2.46

Grizzly Gulch 188 3.65 3.1

TOTALS 830 acresTotal: 23.5 billion px

Avg: 4.7 billion pxAvg: ~3.2 px/cm2

(Area Estimates: http://www.earthpoint.us/Shapes.aspx)

Funding sources and many thanks to…TimeScience / www.time-science.com• Christopher Zimmermann (Data management, image processing, online interface)

University of Chicago• Justin Borevitz (U. Chicago, not at ANU)• Nina Noah, Whitney Panneton (University of Chicago)

GigaPan Systems – http://Gigapan.org

Download this talk here:http://bit.ly/ESA2012