Electronic Conducting System

Kenzo AbrahamsSupervisor: Mehrdad GhaziasgarCo-supervisor: James ConnonMentored by: Diego Mushfieldt

OverviewIntroductionUser Interface SpecificationHigh Level DesignLow level DesignDemo

IntroductionInteractive Conducting SystemTracks hands using webcamReal-time alterations depending

on hand gestures◦Change volume◦Change tempo

User Interface Specification Graphic User Interface (GUI)User interacts using webcam and

mouse

User Interface Specification

User Interface Specification

User Interface Specification

High Level DesignThe solution can be broken up

into 3 parts◦Input ◦Image processing◦Adjust music

Low Level Design

Low Level DesignInput

◦The frames need to be acquired from the webcam

◦cvQueryFrame(capture)

Low Level DesignImage Processing

◦Convert copies of frames to HSV colour space

◦cvCvtColor(frame, img_hsv, CV_BGR2HSV)

Low Level DesignImage Processing

◦Skin segmentation◦Use of a predefined method◦Detect if pixels in the rectangle fall

within a certain range

Low Level DesignImage Processing

◦Set hand as region of interest

◦cvSetImageROI(pHueImg, pHandRect )

Low Level DesignImage Processing

◦Perform Camshift

cvCamShift( pProbImg, prevHandRect, cvTermCriteria( CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 10, 1 ), &components, &HandBox )

Low Level DesignAdjust Music

◦Library RTcmix is used to produce the music

◦Can be imbedded into C++ code◦load("WAVETABLE")◦Wave = maketable("wave", 1000,

"tri")◦WAVETABLE(start time, duration,

amp, frequency, pan, wave)

DemoOverview of demonstration

◦Track the right hand◦Perform the gestures that are going

to be used◦Track the left hand◦Perform simple gestures such as

raising and lowering hand◦Move each hand separately

Project PlanGoal Due Date

Learn how to use OpenCV and its tools.Elicit the requirements and define a designers interpretation of the problem.

CompletedFrom the users requirements design a prototype for the system

CompletedConstruct the system with all its functionality present Term 3Test the system and deployment Term 4

References O'Niel, D L. (2008). Music Theater Jobs. Available:

http://www.ehow.com/list_6547349_music-theater-jobs.html. Last accessed 29th March 2012.

Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. pp31-87.

Ivanciuc, O. (2005). SVM - Support Vector Machines . Available: http://www.support-vector-machines.org/. Last accessed 29th March

Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C. (2009). The USB Virtual Maestro: an Interactive Conducting System, pp.250-255, NIME2009

Borchers, J., Hadjakos, A., M¨uhlh¨auser, M. (2006), MICON a music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, pp254–259.

Questions and Answers