Date post: | 18-Nov-2014 |
Category: |
Self Improvement |
Upload: | chad-udell |
View: | 1,310 times |
Download: | 2 times |
Hacking Robots for Fun and Profit
presented by Chad Udell at Flash and the City, 2010
• Flash is eminently hackable and ideal for building UIs for experiences that integrate with hardware
• Tons of devices are coming with web connections
• Hardware is getting cheaper and easier to get
• Hobbyist techniques can be adapted for real world
Hacking Robots for Fun and Profit
A History of Hacking Flash with Hardware
• Planet Explorers: 13 Experiences
• X-Rovers are one of 7 interactive pieces
About the Adler Planetarium Exhibit Project
• What is it?
• How does it work?
• What is the experience?
• Demos
• Code Samples
Planet Explorers: X-Rovers
• Emulating a Mars Rover like experience
• Has to have a “mission”
• Has to be easy to use
What is it?
• Origins (KUD)
‣ Remote Exploration is the Concept
• Key Points
‣ No Crying Children
‣ Some Learning
‣ Ages 3-8
Definition of the Experience
• Custom R/C
First Thoughts
• T1
• Imperial Droid
• Skillset - We aren’t robotics engineers or electricians
• Budget - Every client is conscious of this - duh
• Scope - Need to integrate a timer, controls, basic computer vision, long term maintenance solution
• Quality - Uptime is crucial. “Out of Order” is not an option
Basic Limitations
Choosing a Robot
• Essentially we need a connected webcam
• Needs to be cheap
• Needs to be hackable
Enter, Rovio.
A Diagram of the system
!"#$%&'()*+,-./+*0)
UPS (APC BR1200)
Happ Controls
2 Stations, each containing:
Happ Kiosk Amp and Speakers
Netgear Wireless Access Point WG-102
Rovio
APC AV C2
Rovio Charger1"234&+56(!11(.7
Switch
Server (Red 5)
• Skillset - Essentially like building a mashup (documented API, built in webserver)
• Budget - They are inexpensive and readily available (important for uptime)
• Scope - many of the requirements are handled in some built in fashion already (wayfinding, charging).
• Quality - We have a company backing it
Rovio to the Rescue
• Skillset - We like it.
• Budget - No extra expensive hardware to be designed, no expensive proprietary SDKs.
• Scope - Still some questions here. Going to have to manage the client’s expectations.
• Quality - Debugging and remote config is easy
Flash to the Rescue
• Rovio responds via HTTP (CGI script)
• Use of key listeners to trigger API calls.
Controlling the Rovio
/rev.cgi?Cmd=nav&action=value&drive=d_value&speed=s_value Input Parameter
value = 18 d_value = 0 (Stop)1 (Forward)2 (Backward) 3 (Straight left) 4 (Straight right) 5 (Rotate left by speed) 6 (Rotate right by speed)7 (Diagonal forward left) 8 (Diagonal forward right) 9 (Diagonal backward left) 10 (Diagonal backward right) 11 (Head up) 12 (Head down) 13 (Head middle) 14 (Reserved) 15 (Reserved) 16 (Reserved) 17 (Rotate left by 20 degree angle increments) 18 (Rotate right by 20 degree angle increments)s_value = 1 (fastest) – 10 (slowest)
Demo 1: Sending Commands to Rovio
Controlling the Rovio
• Rovio sends via MJPEG via HTTP (CGI script)
• Set a loader’s source to the URL
Getting the Video
Camera Control – GetData.cgi
DescriptionThe basic command for acquiring MJPEGGrammar/ChangeResolution.cgi?ResType=value[&RedirectUrl=sUrl] Input ParameterNonePrivilegeNoneReturn ValueAn instance captured motion image.
• Results were marginal (12-15 fps)
Demo 2: MJPeg streamin a Loader
Getting the Video
Simple color detection -
• http://www.interactionfigure.nl/2009/colour-tracking-with-the-webcam-in-flash/
Color detection in Processing
• http://jamesalliban.wordpress.com/2008/11/16/colour-detection-in-processing/
Haar training for object detection
• http://note.sonots.com/SciSoftware/haartraining.html
Modified facial detection
• http://www.quasimondo.com/archives/000687.php
Deeper Research on Video/Image Processing
• Shopping = Fun
Technology Evaluations
• Reading Electrical Engineer Documents = Not so Fun
• Wiring Boards = Fun for some
• Building PoCs = Super fun!
• Software Proof of Concept
• Hardware Hacking and Integration
Prototype
Happ Config 1
Happ Config 2
Demo 3: Happ Controls
Prototype
• Video was ok, but needed some juice.
• Computer vision was deemed better than AR or object detection
• Play control overall was very pleasing
• Testing, Testing, Testing
Alpha
Play Testing Video
• Transcode the Video to H264 (SteamStream)
• Replace the MJPEG loader with a Netstream object
• Better results. Lower latency.
• Approaching 24-30fps
• There is a bit of a hiccup with the initialization of the stream.
Enter Red 5/Xuggle
Demo 4: Red5/Xuggle
Enter Red 5/Xuggle
• Considered AR
• Too Inflexible - Markers weren’t attractive
• Considered Object Detection (Haar Cascades/OpenCV)
• A tad slow-ish
• Attempted using simple ColorBoundsRect
Processing the video
Terminator Vision
Demo 5: Computer Vision
Finding the Targets
What does that get you? (The Good)
What does that get you? (The Bad)
How about Creating a Unique Target?
Not Unique A Lot More Unique
• Eliminate False Positives (was better, but not perfect)
• Ensure Speedy Detection (was a little slower)
• Settled on Color Detection via Thresholding
• Tomek’s Blob Detection
• http://play.blog2t.net/fast-blob-detection
• Soulwire’s Color Averaging
• http://blog.soulwire.co.uk/code/actionscript-3/extract-average-colours-from-bitmapdata
Processing the video
Demo 5: Revised Computer Vision
Fixing the Video
In Action
In Action
In Action
In Action
In Action
In Action
• Charging handoff
• Locking the kiosks down
• C++ Socket Server for the Lights
• Startup, shutdown
• Training for the switchover
Buttoning it up
• SWF to EXE is not needed anymore. AIR 2!
• Test early! Test Often! Test Under lots of Conditions!
• Static Electricity Sucks (installation is crazy)
• Your Kiosk Can Never Be Too Stable (Bullet Proof Your Kiosks)
• Logging, Logging, Logging
• Buy Lots of Extras
Lessons Learned
• Security/WiFi Cameras - http://www.axis.com
• Tons of X10 Stuff - http://www.smarthome.com
• Other Robots - http://www.spykeeworld.com/
• Rocket Launcher - http://www.thinkgeek.com/geektoys/warfare/8bc4/?cpg=cj
• Ultimarc Controls - http://www.ultimarc.com/
Other stuff to hack!
• http://electricpineapple.net/ - Erik Peterson’s blog
• http://ionagroup.com/labs - Iona’s Blog
• http://visualrinse.com - My Blog
Questions?
More info, code samples, etc.
Thank you
THE IONA GROUP 620 West Jackson Street • Morton, Illinois 61550
P: 309.263.4662 • F: 309.263.8262 • 888.644.IONAwww.ionagroup.com
• Find Us on: