Date post: | 04-Aug-2015 |
Category: |
Engineering |
Upload: | mattiastiger |
View: | 38 times |
Download: | 3 times |
Kitchen OccupationTSBB11
2013
Problem
• Students need student kitchens to heat food.• Students want to be able to find the least occupied
kitchen.• The university wants to make informed decisions on
where and if more kitchens are needed.
The Group (main contributions)• Mattias Tiger – Project Leader, Software Architecture• Gustav Häger – System Integration, embedded• Erik Fall – Tracking, counting• Malin Rudin – Evaluation• Alexander Sjöholm – User and sensor interfaces• Nikolaus West – Quality control, queue detection• Martin Svensson – Documentation, system I/O
System overview
• Counts entries and exits to room• Estimates severity of potential queues• Communicates to server via HTTP
System overview
Kinect Kinect
Room Entrance
Computer
System Overview
• Highly configurable • Simple to Calibrate• Runs with and without GUI• Easily extendable– We ended up creating a great, reusable,
architecture for doing computer vision.
Initial solution
• Initial plan to use network cameras and process on a server
• Used a background model with basic tracker• Occlusion was a large problem
Changing circumstances
• Could not send data over the network• Needed to process on the spot (on a
cheap/slow computer)• Background model needs to process each
pixel, a bit slow• A new approach was needed.
New approach
• Needed some way to segment humans• Idea to detect only heads, or only movement• Optical flow for movment• Hough circles for heads
Tested approaches – hough circles
• Use hough circles to detect heads in the image
Tested approaches – hough circles
• Hough circle method is not robust enough
Tested approaches
• Segmentation based on optical flow• To computationally heavy• Not fast enough for a cheap computer
New approach again – depth data
• Tried using depth data• Stereo is hard to calibrate, and slow to
calculate• Very easy and cheap to aquire using a kinect
Image processing
• Human segmentation• Depth image from Microsoft Kinect camera• Simple assumptions
Image processing
Image processing
• Occlusion handling
Tracking
• Segmentation not enough for solving our problem, need some approach for counting
• How to track appearing objects in a robust way
• Follow objects from one frame to another – Pairing closest objects
Main error sources
• Objects randomly appearing, outliers– Maximum pairing distance limits
• Occlusion– Keep lost objects alive
• Noise, flickering objects – Minimum lifetime before considered human
counted
Counting
• Registrating entries & exits– Door area• New object Entries + 1• Object lost Exits + 1• Not enough
– Checkpoints • Circle lines
– Removes some false entries & exits
Queue Severity Estimation
• What is a queue?• People traveling slowly along the same path• Position, speed and movement factor in.
Queue Severity Estimation
• Cubic Beziér Splines completely defined by:– Two endpoint positions (person positions)– Velocities at endpoint (person velocities)– Curve parametrization (relative weight between
velocities and positions)
Queue Severity Estimation• People connected by short spline in same
queue• Proportion of frames with visible queues
determines queue severity
System Setup
• Two steps– Calibration– Configuration
• Target user computer vision skills– None
Calibration
• What is the current height of the sensor?– Calibrate!
Configuration
• Just listen tothe manual• Specify – Doors– Checkpoints– Exclusions
Configuration
Dense Debugger
• Where did it go wrong?– Intuitive debugging
• Speed is crucial– Built-in profiling
• Talk to me– Verbose logger
Dense Debugger
The Grid
System Input
• Initially only from pre-recorded data• Need to run from network cameras– Control multiple sensors at once
• Desire to manage different sensor types– Enable easy integration of other sensor types
System Input• Final supported video input– Any OpenCV-compatible video files– Any OpenCV-compatible cameras– Microsoft Kinect• RGB• Depth
• Developed with modularity and extendability taken into account
System Output
• Transmit desired information using HTTP– Make it easy to change output information– Possibility to easily switch target servers
System settings
• Loads of different parameters in the code– Some algorithms have more than 10 parameters– Some parameters are calibrated using the GUI• Height threshold• Queue sensitivity
– Camera types and network camera locations
System settings
• One configuration file– Easy to tune different parameters– Good portability– Tune using GUI and then switch to the embedded
version• Automatically writes all settings to file after
program termination.
System settings
Example file with required parameters.
Software architecture
• Cross platform• Modular– Extendable– Reusable
• Configurable• Ideal for rapid prototyping & testing
PerformanceData sets
PerformanceData sets
PerformanceData sets
Performance
Evaluation metric
Performance
Queue severity
Results
Sequence name Total number of entered (GT)
Ain Total number of exited (GT)
Aout Length
R-Kitchen 108 (108) 100% 101 (104) 97% 32 min
U-Kitchen 123 (122) 99% 134 (135) 99% 31 min
B25-Kitchen 131 (141) 93% 82 (91) 90% 30 min
Results
R-Kitchen: Number of entries (esitmated and ground truth).
Results
U-Kitchen: Accuracy of entries.
Results
B25-Kitchen: Number of exits (estimated and ground truth).
The final product• A cheap and accurate system for monitoring room
usage• A very general purpose image processing pipeline• Total configuration without re-compiling• Great debug and installation user interfaces• Support for built-in, automatic evaluation of
algorithms
Questions?