January 14, 2014 Sam Siewert
ECEN 5043
Computer and Machine Vision
Lecture 1 – Introduction
Part-2
Biological Vision vs. Machine Vision (Why A Honey Bee is Better than HPC for CV)
Humans - 100 million
Photoreceptors – 10 billion Neurons (Cerebral Cortex)
– Brain with 100 billion Neurons
– Millisecond Transfer
– Massively Parallel Analog + Digital Computation
Synapse Match is a Challenge – 7000 Connections from 10 Billion Neurons
– 3 Year Olds Have 1015 Synapses
CPU to Digital Camera/HDD – Connects 10’s of millions of pixels
– to Several Billion transistors
– Through Sequential Logic and I/O Bus
Sam Siewert 2
960K Neurons in flight:
Learns locations,
complex odors,
colors, and shapes;
with high efficiency
(500 Watt/Kg), 0.218g
Brain plasticity for
learning,
connectedness,
concurrency,
integrated sensing,
power efficiency,
and resiliency
2012 – 8 billion?
NVIDIA GK110
28nm, (7.1 billion)
Intel MICA 22nm
(5 billion)
http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2011.svg
How We’ll Do It Assessment of Theoretical Learning
– Two Mid-term Exams (1/2 way, 7/8 way)
– FINAL
Practice – 5 Labs
Application – 1 Extended Lab with your Own Design
Sam Siewert 3
Linux Lab and Desktop Options
Native Linux Installation – Ubuntu
Logitech C200 or C270 Camera(s)
OpenCV
ffmpeg
GIMP
Transformer.uaa.alaska.edu – available to all remotely
and in A219
Virtual-Box Ubuntu Installation
Beagle xM Ubuntu, Intel Terasic Atom Yocto Linux
Sam Siewert 4
Sam Siewert 5
Administrivia Lectures – PowerPoint with Camtasia – Recorded on Wednesdays in ENGR 227C, Distributed via Blackboard by Thurs Morning Introductions – Instructor (Office Hours) -
http://www.cse.uaa.alaska.edu/~ssiewert/Schedule-Spring-2014.pdf – Students (Introductions) – Let’s all join Google+ Circle (I will create and
invite you)
UAA Blackboard – http://www.uaa.alaska.edu/classes/
Personal Lab – You MUST Have Native Linux and I recommend VB-Linux Too – Either using your own Laptop – Or Using A219 Lab at UAA
UAA Beagle xM Linux Lab – A219, http://www.cse.uaa.alaska.edu/~ssiewert/cpal.html
January 14, 2014 Sam Siewert
Linux Digital Video and CV
Processing Skills
Introduction Session
Basic Lab Observations
CV is Compute Intensive
– Lower Resolution and Frame Rates (e.g. 640x480 or 320x240 at
30Hz)
– High-End is Really Intense (HPC) – E.g. 1000 Hz 4K Cameras
like http://www.idtvision.com/, or http://www.photron.com/
– Humans Seem to Saturate at 60Hz (Current Theory)
– 60Hz Stereo in HD is still a Massive Data Rate (1920x1080 x 3
bytes x 60 x 2), or about 720 MB/sec!!
We will work at Low Resolution and 30Hz, but with Both
2D and 3D
Both Binocular 3D, and RGB-Depth
Sam Siewert 7
Tutorial CV Papers – IBM DeveloperWorks
Build a compute node or small cluster application
and scale with HPC -
http://www.ibm.com/developerworks/cloud/library/cl-
cloudscaling1-hpcondemand/index.html
Explore video analytics in the cloud -
http://www.ibm.com/developerworks/cloud/library/cl-
cloudscaling3-videoanalytics/
Machine data analytics -
http://www.ibm.com/developerworks/library/bd-
mdasecurity/index.html
Sam Siewert 8
Labs
I will POST to BB and External Website on Thursdays
Read, Review, Start and Question that Weekend
Bring Questions to Office Hours Mon, Tues, Wed the Following Week
Lab Due one Week Later
This Works Great if YOU Keep Up
I will POST Lab #1 on 1/15/2014, Due on 1/26 for Full Credit, Accepted Late Until 1/30 (10% Penalty)
Sam Siewert 9
January 14, 2014 Sam Siewert
OpenCV Demos
Overview Session – Passive Computer
Vision Methods
2D & 3D Passive Computer Vision
Sam Siewert 11
Analog
Camera #1
LEFT (NIR, Visible) Linux with
OpenCV (x86, TI OMAP, Atom)
USB 2.0, PCIe
Host Channels Storage
Analog
Camera #2
RIGHT (NIR, Visible)
Linear Hough Transform
2D Skeletal Transform
3D Disparity & Depth Map Canny Edge Finding
Face Detection/Recognition
January 14, 2014 Sam Siewert
OpenNI
Overview Session – Active Computer
Vision Methods
3D Active Computational Photometry
Sam Siewert 13
Analog
Camera #1
RGB (Visible)
Altera FPGA
CVPU (Computer Vision
Processing Unit)
Mobile
Sensor Network
Processor (TI OMAP, Atom)
Networked
Video
Analytics
HD Digital
Camera Port
(Snapshot)
USB 2.0, PCIe
Host Channels
Flash
SD Card
Analog
Camera #2 (Near
Infrared)
TI DLP Light-crafter Kit http://www.ti.com/tool/dlplightcrafter
Depth Map
IR Pattern Projection
https://www.cs.purdue.edu/homes/aliaga/cs635-10/lec-structured-light.pdf
Photo credits and reference:
Dr. Daniel Aliaga, Purdue University
https://www.cs.purdue.edu/homes/aliaga/
3D Computer Vision Transforms Long Range ( > 5 meters) Using Passive Binocular Methods – Impractical to Project from a UAV or Long Range Observer
– Requires Image Registration
– Accurate Camera Intrinsic (Camera Characteristics) & Extrinsic (e.g. Baseline)
Short Range ( < 5 meters), Structured IR Light Projection for RGB-D – Compare to ASUS Xtion and PrimeSense – Off-the-Shelf
– Robust Depth Maps with Less Noise
– Showing Significant Promise to Improve CV Scene Segmentation and Object Recognition Compared to 2D
– “Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, December 2013.
Sam Siewert 14
Noise in Passive Depth Maps
Robust Active Depth Map
“Change Their Perception”, By Xiaofeng Ren,
Dieter Fox, and Kurt Konolige, IEEE RAS,
December 2013.
Off-The-Shelf RGB-Depth Mappers
Intel Creative Camera – Windows Perceptual SDK
ASUS Xtion Short and Long Range – OpenNI
PrimeSense (Kinect Old and New) – MS SDK, ROS
Sam Siewert 15
Discrete Convolution
Sam Siewert 16
a[1,-1] a[0,-1] a[-1,-1]
a[1,0] a[0,0] a[-1,0]
a[1,1] a[0,1] a[-1,1]
Summary
Numerous MV and CV Applications – Inspection and Process Automation – MV Domain
– Interactive Systems and Augmented Reality – CV Domain
– Robotics – MV and CV
– Study of Human Vision and Vision Prosthetics – CV
2D Image Processing (Machine Vision) – Capture, Enhancement, Segmentation, Recognition
Passive 3D Computer Vision – Stereo Capture, Calibration, Enhancement, Registration, Depth
Mapping, Segmentation, Recognition
Active 3D Machine Vision (It’s Cheating!) – Structured Light Illumination and IR/Visible Capture, IR Analysis and
Depth Mapping, Visible Image Registration
– Works Between 0 and 5 Meters Well
Sam Siewert 17