A Project Report

On

Air String And Air Pong

Submitted in the Partial Fulfillment of the

Requirement for the award of

Bachelor of Technology

in

Electronics & Communication Engineering

By

Ramesh Agarwal (20085094)

Manish Singh (20085027)

Ravi Pratap Gond (20085061)

Ashwarya Pratap Singh (20075087)

Under the guidance of

ASIM MUKHERJEE

Assistant Professor

Department of Electronics & Communication Engineering

Motilal Nehru National Institute of Technology Allahabad

Allahabad – INDIA

0

Department of Electronics & Communication Engineering Motilal

Nehru National Institute of Technology Allahabad

Allahabad – INDIA

CERTIFICATE

This is to certify that the work contained in the thesis titled “Air String And Air

Pong” submitted by Ramesh Agarwal, Manish Singh ,Ravi Pratap Gond and

Ashwarya Pratap Singh in the partial fulfillment of the requirement for the award of

Bachelor of Technology in Electronics and Communication Engineering to the

Electronics and Communication Engineering Department, Motilal Nehru National

Institute of Technology, Allahabad, is a bonafide work of the students carried out

under my supervision.

Date:

Place:

Mr . Asim mukherjee

Asst.professor

ECE Department

MNNIT, Allahabad

i

ACKNOWLEDGEMENT

It is a great privilege for us to express our deep sense of gratitude to our supervisor

Mr.Aseem Mukherjee of Electronics and Communication Engineering Department,

Motilal Nehru National Institute of Technology Allahabad for his stimulated guidance

and profound assistance. We shall always cherish our association with him for his

constant encouragement and freedom to thought and action that herendered to us

throughout our term paper project. We also feel a great pleasure to thank one and all

that helped us in carrying out this project.

Date:

Place:

Ramesh Agarwal (20085094)

Manish Singh (20085027)

Ravi Pratap Gond (20085061)

Ashwarya Pratap Singh (20075087)

ii

ABSTRACT

Our project is Air String, a synthesized string instrument that can be played in real

time by waving fingers with bright green color tips in motion of stroking a string in

front of a camcorder. Our implementation is based on Karplus Strong algorithm.

We got the idea for the product Microsoft Kinect of Xbox 360. The original idea

was to implement a synthesized string instrument that can be played in the air similar

to Air Guitar.The concept of the project is to provide a user interface similar to that of

playing the harp except for the fact that there is no physical instrument in front of the

user. Instead, the user’s finger motion in the air is recorded in real time to play virtual

strings of different notes. We liked the idea of combining the visual component

(VGA) and the audio component (audio codec) together.

iii

SUMMARY

For the project, we used Spartan 3E Platform FPGA Startar Kit,a XST 8.0 MP

Webcam,a HCL VGA compatible monitor at a 640x480 resolution (connected to the

Spartan 3E board via VGA Video Port) and a Stereo Speaker. We keep track of the

movement of a player’s fingers to decide whether to play a string, which musical note

of a string to play and whether to play the same note for the second time. We can

detect the movements of fingers by detecting the changes in RGB values of pixels on

the screen. For that, a player should put bright green color marker caps on her/his

finger tips (or wrap the fingers with color tapes). The monitor and the webcam face

the player while the webcam shoots a video of the player’s finger movements. The

monitor screen shows where the fingers are without a mirror effect along with white

lines and letters in the background. The lines indicate the each section for different

notes and the letters tell the player which note the section is allocated to.

iv

TABLE OF CONTENT

Page No.

CERTIFICATE i

ACKNOWLEDGEMENT ii

ABSTRACT iii

SUMMARY iv

CHAPTER 1: INTRODUCTION

1.1 Introduction and Task Analysis 1

CHAPTER 2: REVIEW OF ANALOGOUS CIRCUIT 4

CHAPTER 3:DESIGN AND IMPLEMENTATION 6

3.1 Hardware 6

3.2 Embedded Web Server Protocol Analysis 7

3.3 TCP/IP Stack 8

3.4 HTTP Protocol 14

CHAPTER 4:COMPOSITION OF ELECTRICAL CIRCUIT 17

4.1 Microcontroller 17

4.1.1 Pin Description 18

4.1.2 I/O Ports 21

4.2 ENC28J60 Ethernet Controller 22

4.2.1 Overview 22

4.2.2 Features 23

4.2.3 Pin Configuration 24

CHAPTER 5: MODEL INVESTIGATION 25

CHAPTER 6: CONCLUSION 29

CHAPTER 7: REFERENCES 30

v

APPENDIX A : LISTING OF CODE FILES 31

APPENDIX B : PROTEUS ISIS SCHEMATIC 32

APPPENDIX C: COST 3

LIST OF FIGURES

SERIAL NO FIGURE NAME PAGE NO

vi

2.1.1 KARPLUS ALGORITHM CHART 3

2.1.2 GRAPH BETWEEN VOLTAGE AND INPUT 4

2.2.1 FREQUENCIES OF MUSICAL NOTES 5

3.1.1 VGA MONITOR FOR AIR PONG 6

3.1.2 CIRCUIT DIAGRAM FOR VGA 6

3.1.3 CONNECTION INSIDE VGA 7

3.1.4 VGA SIGNAL TIMING 7

3.2.1 VGA TIMING WAVEFORMS 9

4.4.1 DIFFERENT NOTES OF GUITAR CORDS 16

5.1.1 RAY DIAGRAM FOR SSSM 18

5.2.1 STSM 20

5.3.1 FREQUENCY REGULATION CHART 21

5.3.2 FREQUENCY REGULATION WAVEFORM 22

vii

viii

CHAPTER 1

1.1 INTRODUCTION AND TASK ANALYSIS

Our project is Air String, a synthesized string instrument that can be played in

real time by waving fingers with bright green color tips in motion of stroking a string

in front of a camcorder. Our implementation is based on Karplus Strong algorithm.

We got the idea for the product Microsoft Kinect of Xbox 360. The original

idea was to implement a synthesized string instrument that can be played in the air

similar to Air Guitar.The concept of the project is to provide a user interface similar

to that of playing the harp except for the fact that there is no physical instrument in

front of the user. Instead, the user’s finger motion in the air is recorded in real time to

play virtual strings of different notes. We liked the idea of combining the visual

component (VGA) and the audio component (audio codec) together.

For the project, we used Spartan 3E Platform FPGA Startar Kit,a XST 8.0 MP

Webcam,a HCL VGA compatible monitor at a 640x480 resolution (connected to the

Spartan 3E board via VGA Video Port) and a Stereo Speaker. We keep track of the

movement of a player’s fingers to decide whether to play a string, which musical note

of a string to play and whether to play the same note for the second time. We can

detect the movements of fingers by detecting the changes in RGB values of pixels on

the screen. For that, a player should put bright green color marker caps on her/his

finger tips (or wrap the fingers with color tapes). The monitor and the webcam face

the player while the webcam shoots a video of the player’s finger movements. The

monitor screen shows where the fingers are without a mirror effect along with white

lines and letters in the background. The lines indicate the each section for different

notes and the letters tell the player which note the section is allocated to.

For prospective players of our virtual instrument, now we explain how

to interact with our program and play a song. Our program is set to look for a bright

green color as a virtual stroker (plucker) of a string. Wear a green marker cap or tape

on your finger. Face the monitor and the webcam, so you can see where your fingers

are and which sections on the screen to aim to play a note of your choice. For

example, to play middle C, aim for and cover the section labeled middle C on the

screen with the green marker cap or tape. If you want to play the same note for the

second time consecutively, you need to uncover the section by moving the finger

away from it and place the finger in the section again covering it with green. To play

a note, you can waive your fingers back and forth or move them horizontally to cover

2

CHAPTER 2: High Level Design

2.1Karplus Strong Algorithm

We employed Karplus Strong algorithm to implement a string. This algorithm is

surprisingly simple yet works very well. For a piano, two or three strings are used per

note. Since our user interface (plucking or stroking rather horizontally) is quite

different from that for a piano (striking down vertically), we construct one string per

note and the sound we synthesizes is closer to that of a guitar than to that of a piano.

Wikipedia definition of Karplus Strong string synthesis is a method of physical

modeling synthesis that loops a short waveform through a filtered delay line to

simulate the sound of a hammered or plucked string or some types of percussion.

The actual implementation of the algorithm for our project is depicted in the diagram

below:

FIGURE NO- 2.1.1

The hardware components to implement a string consist of a shift register, a phase

shifter and a low pass filter. The basic concept here is that an input pulse goes through

a certain length of shift register for a coarse tuning and it goes through a phase shifter

for a fine tuning. Then the output from the phase shifter goes through a simple low

pass filter which adds a delayed version (previous output) to the output and divides

3

the total by 2. The simple low pass filter basically averages two values before the

result is fed back into the shift register. For an input pulse, we chose a saw tooth wave

with a sharp raise at the beginning. This works very well for a nice string sound and it

works much better than some white noise. The amplitude of the pulse is 1, and the

step values of the pulse to be fed into the shift register are converted to a 3.17 number

format that is used in our hardware design. For example, 0.5 in 3.17 format is

represented as 0.5*2^17 = 20.65536. For your reference, mathematical representation

of the phase shifter is y(n) = {x(n) - y(n-1)} * η + x(n-1).

FIGURE NO-2.1.2

4

2.2 Frequencies of musical notes that we implemented are

tabulated below:

.

FIGURE NO-2.2.1

The length of the shift register and the value of η are related to the pitch (frequency)

of a note. Our default sampling rate, fs, is 8,000Hz. The length of a shift register, N,

for a note of frequency, fo, can be obtained from the equation fs/fo = N . For middle C,

the length of shift register that we need is then 8000/261.626 = 30.58. We set N equal

to 30. The sample delay, Δ, for middle C is then 0.58 and is defined in terms of η as

Δ=(1-η)/(1+η). Conversely, η=(1-Δ)/(1+Δ). The low pass filter uses a sign extended

right shift for damping (decay factor). 0.5 is the maximum value for damping.

However, we slightly lowered the decay factor to 0.4921875 because this produced

much better string sounds than when damping was 0.5.

5

CHAPTER 3: VGA__________________________________________________________________________________

3.1 Driving a VGA monitor for Air Pong

A VGA monitor requires 5 signals to display a picture:

R, G and B (red, green and blue signals).

HS and VS (horizontal and vertical synchronization).

FIGURE NO-3.1.1

The R, G and B are analog signals, while HS and VS are digital signals.

The Spartan®-3E FPGA Starter Kit board includes a VGA display port via a DB15

connector. Connect this port directly to most PC monitors or flat-panel LCDs using a

standard monitor cable. As shown in Figure 6-1, the VGA connector is the left-most

connector along the top of the board.

FIGURE NO-3.1.2

6

FIGURE NO-3.1.3

The Spartan-3E FPGA directly drives the five VGA signals via resistors. Each color

line has a series resistor, with one bit each fo r VGA_RED, VGA_GREEN, and

VGA_BLUE. The series resistor, in combination with the 75Ω termination built into

the VGA cable, ensures that the color signals remain in the VGA-specified 0V to

0.7V range. The VGA_HSYNC and VGA_VSYNC signals using LVTTL or

LVCMOS33 I/O standard drive levels. Drive the VGA_RED, VGA_GREEN, and

VGA_BLUE signal s High or Low to generate the eight colors shown.

FIGURE NO-3.1.4

VGA signal timing is specified, published, co pyrighted, and sold by the Video

Electronics Standards Association (VESA). The following VGA system and timing

7

information is provided how the FPGA might drive VGA monitor in 640 by 480

mode.

3.2 Signal Timing for a 60 Hz, 640x480 VGA Display

CRT-based VGA displays use amplitude-modulated, moving electron beams (or

cathode rays) to display information on a phosphor-coa ted screen. LCDs use an array

of switches that can impose a voltage across a small amount of liquid crystal, thereby

changing light permittivity through the crystal on a pixel-by-pixel basis. Although the

following description is limited to CRT displays, LCDs have evolved to use the same

signal timings as CRT displays. Consequently, the following discussion pertains to

both CRTs and LCDs.

Within a CRT display, current waveforms pass through the coils to produce magnetic

fields that deflect electron beams to tr ansverse the display surface in a raster pattern,

horizontally from left to right and vertically from top to bottom. As shown in Figure ,

information is only displayed when the beam is moving in the forward direction—left

to right and top to bottom—and not during the time the beam returns back to the left

or top edge of the display. Much of the potential display time is therefore lost in

blanking periods when the beam is reset and stabilized to begin a new horizontal or

vertical display pass.

8

FIGURE NO-3.2.1

The display resolution defines the size of the beams, the frequency at which the beam traces across the display, and the frequency at which the electron beam is modulated.

Modern VGA displays support multiple display resolutions, and the VGA controller dictates the resolution by producing timing signals to control the raster patterns. The controller produces TTL-level synchronizing pulses that set the frequency at which current flows through the deflection coils, and it ensures that pixel or video data is applied to the electron guns at the correct time.

9

3.3 VGA Signal Timing

The signal timings in Table are derived for a 640-pixel by 480-row display using a 25

MHz pixel clock and 60 Hz ± 1 refresh. Figure 6-3 shows the relation between each

of the timing symbols. The timing for the sync pulse width (TPW) and front and back

porch intervals (TFPand TBP) are based on observations from various VGA displays.

The front and back porch intervals are the pre- and post-sync pulse times. Information

cannot be displayed during these times.

FIGURE NO-3.3.1

10

3.4 Frequency generator

A monitor always displays a picture line-by-line, from top-to-bottom. Each line is drawn from left-to-right.That's hard-coded, you cannot change that.

But you specify when the drawing starts by sending short pulses on HS and VS at fixed intervals. HS makes a new line to start drawing; while VS tells that the bottom has been reached (makes the monitor go back up to the top line).

For the standard 640x480 VGA video signal, the frequencies of the pulses should be:

Vertical Freq (VS) Horizontal Freq (HS)

60 Hz (=60 pulses per second) 31.5 kHz (=31500 pulses per second)

To create a standard video signal, there is more details to take care of, like the duration of the pulses and the relationship between HS and VS.

3.5 Our Video generator

Nowadays, VGA monitors are multisync, so can accommodate non-standard frequencies - no need to generate exactly 60Hz and 31.5KHz anymore (but if you are using an old (non-multisync) VGA monitor, you'll need to generate the exact frequencies).

Let's start with X and Y counter.

CounterX counts 768 values (from 0 to 767) and CounterY counts 512 values (0 to 511).

Now, CounterX is used to generate HS, and CounterY to generate VS. Using a 25MHz clock, we get 32.5KHz for HS and 63.5Hz for VS. The pulses need to be active long enough for the monitor to detect them. Let's use a 16 clocks pulse (0.64µs) for HS and a full horizontal line length pulse for VS (768 clocks or 30µs). That's shorter than what the VGA spec calls for but works fine anyway.

11

We generate the HS and VS pulses from D-flipflop (to get glitch free outputs).

The VGA outputs need to be negative, so we invert the signals.

Finally we can drive the R, G and B signals. As a first cut, we can use some bits of the X and Y counters to get nice square color patterns.

. and we get a picture on the VGA monitor!

CHAPTER 4: USING THE VGA__________________________________________________________________________________

12

4.1 Drawing a useful picture

The sync generator is best rewritten to be used as an HDL module where we generate R, G and B outside. Also the X and Y counters are more useful if they start counting from the drawing area.

Now we can use it to draw a border around the screen.

13

4.2 Drawing a paddle

Let's use a mouse to move the paddle left and right on the screen.

Now that "PaddlePosition" value is known, we can display the paddle.

4.3 Drawing the ball

The ball needs to move around the screen, and bounce back when it touches an object (border or paddle).

First we display the ball. It is a square 16x16 pixels. We activate the drawing of the ball when CounterX and CounterY reach its coordinates.

Now for the collisions. That's the difficult part of this project.

14

We could check the coordinate of the ball against each object on the screen and determine if there is a collision. But that would become quickly a nightmare as the number of objects increases.

Instead we define 4 "hot-spots" pixels, one in the middle of each side of the ball. If an object (border or paddle) redraws itself at the same time that the ball draws one of its "hot-spot", we know that there is collision on that side of the ball.

And finally we can draw all that together.

15

4.4 Screen Arrangements for Air String

For a user-friendly interface, we printed on the monitor screen white lines that outline sections and letters that correspond to notes of the sections. To create this background image that is printed on the monitor all the time while a player is playing, we used Paint program and created a black and white .bmp file as below:

FIGURE NO-4.4.1

Now the background image is stored in the memory and we have two images to send

to the monitor screen: one from the camcorder and one in the memory. We checked

the color value in the memory and if it was 0, black, then we chose the RGB value

from the camcorder for a pixel, otherwise we chose the one bit value from the

memory because the pixel was of white lines and letters. (A simple mux does the

trick.)

16

CHAPTER 5 :Hardware Design

5.1 String Synthesizer state machine (SSSM)

This state machine consists of a shift register, and the phase shifter and the low pass filter are implemented in combinatory logic.

To generate 16 notes of different frequencies, we set the default sampling rate at 8,000 Hz, which is the clock rate for all the state machines. Thus each state machine has a shift register of a different length and a different sample delay value for each phase shifter. A shift register consists of many 20-bit registers. (For better accuracy, we used 3.17 format instead of 3.13 format.) We chose not to use M4K blocks and it worked out well for us because 1) it turned out that we could not afford to add one more clock cycle to access the memory in the string synthesizer state machine because the string trigger state machine (STSM) at a much faster clock (VGA_CLK at 27 MHz, 3375 times faster) has to wait for the SSSM to send a signal before it can move onto the next state (this signaling between two state machines is explained later in details) and 2) we used 70% of M4K blocks to store the background image later. Overall we used about 50% of logic elements and resources available on the board to implement the entire system.

Basically, SSSM works as follows:

if (the string is plucked by STSM){        initialize the shift register with a saw tooth input pulse;         send signal to STSM that the string is plucked (set a flag to 1);         go to state 0; }else{  state 0:         shift register values in the shift register (one right shift);         send signal to STSM that the string is being played (set a flag to 0);         go to state 1;   state 1:         update values for combinatory logic;         go to state 0; }

State machine diagram is shown below:

17

FIGURE NO -5.1.1

5.2 String Trigger state machine (STSM)

This state machine checks for the presence of bright green color in a particular section for a corresponding note by counting a number of pixels, RGB value of which is specified as G > 10'h99 AND R < 10'h80 AND B < 10'h80. Each section for a note has 100x60=6,000 pixels and if there are more than 1,500 pixels whose RGB values meet the above requirement, then the state machine triggers an SSSM that plays a corresponding note.

STSM and SSSM signal each other (handshaking) so that they can coordinate their executions although they work under different clock rates. This way, when a user places a green marker cap in a section for a note on the screen and does not remove it

18

for a while, the string plays only one time and rests until the cap is removed from the section and placed there again for the second consecutive stoke.

Basically, the way STSM works is as follows:

state 0:        if (VGA x and y coordinates are pointing to pixels in the section for this STSM)        {            count the number of bright green pixels;             go to state 0;        }        else if (VGA x and y coordinates reached the end of screen)            go to state 1;        else            go to state 0;

state 1:        if (more than 1,500 bright green pixels are present)        {                if (the string has not been plucked)                {                    signal the SSSM to feed in an input pulse to the shift register;                    set a flag to 1 to remember that the string has been plucked once;                    go to state 2;                }                else // there has been no change in the movement of a player since the last pluck                {                    do not signal the SSSM to feed in an input pulse to the shift register;                    set a flag to 1 to remember that the string has been plucked once;                    go to state 4;                }        }        else        {            do not signal the SSSM to feed in an input pulse to the shift register;            set a flag to 0 to remember that the string has not been plucked;            // the string is ready for the next pluck;            go to state 4;        }

state 2:        if (SSSM signaled that the shift register will be initialized with an input pulse)            go to state 3;        else // wait for SSSM to catch up and signal that the string is read to be played.            go to state 2;

state 3:        do not signal the SSSM again to feed in an input pulse to the shift register;        if (SSSM signals that the shift register is done being initialized)

19

            go to state 4;         else // SSSM is not done initializing the shift register            go to state 3;

state 4:        // reset registers for the next VGA screen check        remember whether the string was plucked this time;        set the green color detection counter to 0;        go to state 5;

state 5:        if (VGA x and y coordinates reached the end of the screen)            go to state 0;        else // green color detection always starts from the beginning of the screen.            go to state 5;

Waiting for the SSSM which operates under a slower clock rate to send signals before it could go to the next state, STSM stays in a few wait states for many numbers of VGA screen sweeps (refreshes). It means that we skip many frames without checking for changes in RGB values of a section. However, this does not affect the accuracy of our program because any fast human hand movement is much slower than the VGA refresh rate and the most human eyes cannot discern discontinuity when the frame rate is over 100 FRS (frame rate per second).

State machine diagram is shown below:

FIGURE NO-5.2.1

20

5.3 Frequency regulatorWe needed to bring down AUD_DACLRCK (48 KHz) to 8,000 Hz sampling rate to

clock SSSMs. This frequency regulator simply uses a counter to wait and to generate

a lower frequency clock signal than the input clock signal. Using this frequency

regulator and muxes, we produced three different sampling frequencies, fs, 12 KHz,

8,000 Hz and 4,800 Hz respectively, and a player can choose three different sets of

notes using dip switches on the board. When all the SWs are set to 0, fs is the default

value of 8,000 Hz. If SW[0] = 1, fs is 4,800 Hz. If SW[17] = 1, fs is 12 KHz.

Although they are not separated by one octave exactly, they are all harmonics. This is

caused by the fact that we keep the same value for N in the equation, f s/fo = N, and

vary the value of fs, so the output frequency, fo, is not scalable by a player. However,

a player can enjoy different timbers of a string sound. The background image of white

lines and letters for notes does not change as a different sampling rate from the

default is chosen. Three sets of notes that are synthesized are tabularized below:

FIGURE NO-5.3.1

21

The figure below shows up to what number the counter should count to generate

clock of which frequency.

FIGURE NO-5.3.2

5.4 Screen FlipperWhen we display a video stream from a camcorder on a monitor, a mirror effect is

observed. This confuses a player when s/he faces the camcorder and the monitor to

aim for a certain section on the screen to play a note because as s/he moved her/his

hand from right to left, s/he sees the hand move from left to right. For a user friendly

interface, we corrected the mirror effect by adding Mirror_col module. Like Altera,

terasIC provides tutorial documents and demonstration project files. One of the

project files with a top module named DE2_CCD.v comes with Mirror_col module

that reverses the frame captured from the sensor in TRDB_DC2 camera. We modified

the Mirror_col and added to our project to reverse the screen. Because Mirror_col

22

uses two stack RAM as buffers, the left half side of the screen is actually one frame

behind the right half side of the screen. However, due to the fast VGA clock, this does

not affect the speed of visual (hand movement) that a player sees or audio (string

sound) that a player hears.

CHAPTER 6 :RESULTS__________________________________________________________________________________

6.1 Results of the Design

The Air String has a user friendly interface, and it makes it easy for anyone to play a song. The way it’s played is quite intuitive to most people without any skills in musical instruments. Although the letters for notes are printed on the screen for a quick reference, if a player memorizes the sequence of notes of a song and practices, which most musicians do, s/he can play the song much smoothly, fast and easily.

Unfortunately we did not have more time to add more features to Air String. A user can choose from three different timbers of a string sound, but the notes in each set are not exactly distanced by an octave. Also, the notes with a sampling rate different from the default do not match the corresponding letter on the screen because we have only one background image stored in memory. Using additional memory such as SRAM and re-designing the frequency generator could improve Air String.

A tip for a player: Having a bright yellow light shining over the camcorder from the behind helps the color of the marker cap to be detected as a bright green color easily. It helps the player play better.

To detect the change in the finger movements of a player, we count the number of pixels of a specific color (in our case, a bright green color whose RGB value in hex is 24h009900). To look for this color, we check if an RGB value from the camcorder satisfies the condition (VGA_G > 10'h99 && VGA_R < 10'h80 && VGA_B < 10'h80). We picked a green color of 24h009900 because we usually do not see this color around us so that it could be easily distinguished from any background. Instead of using absolute values for RGB for a color check, when we tried the relevant condition (mGreen > mRed && mGreen > mBlue), the color detection did not work very well thus making it harder for a player to play.

6.2 Trade-offThe way we designed the Air String is that any notes should not keep playing once it was played while a green marker cap stayed in the same section for long. We consider a player meant to play a string when of a section corresponding to a note is filled with bright green color pixels (1,500 pixels out of 6,000 pixels for a section). The number 1,500 seems to work the best for us when there is yellow light over the camcorder. If we lower the number, then a player does not need to place the entire

23

marker cap in a section and it could help increase the speed of finger movement from a note to another. However, this also more easily triggers a string, so sometimes a note plays more than once when a player did not mean to play the note twice consecutively. If we increase the number, then it takes a player a little more time to move from one note to anther away from it although this decrease the chance of a note being played more than once when a player did not mean to.

6.3 Accuracy in terms of timing issuesReversing the screen to remove the mirror effect for a player’s convenience was

tricky because we already had some issue with time, which was cause by signaling

(handshaking) between two state machines of two different clock rates as one state

machine with a faster clock has to wait for the other for many clock cycles while

VGA does not stop refreshing the screen. Air String needs to be played in real time

and there should not be any delay between a player’s finger movement and the video

stream on the monitor screen. Using stack RAMs as buffers to reverse the screen

causes VGA to display the left half side of the screen one frame late. If you have a

really good eye, then you might be able to detect it. However, it did not affect any of

our test players performance.

24

CHAPTER 7: Conclusion and Future Work__________________________________________________________________________________

7.1 ConclusionsOverall, the project was a success. We produced a working model with a user friendly interface. We believe Air String is easy for anyone to learn to play and quite entertaining and educational for children to stimulate their interest in musical instruments. Throughout the project, we had an opportunity to demonstrate our skills that we obtained from previous lab assignments and also had an opportunity to learn much more about FPGA and new things such as TV decoder and Karplus Strong algorithm.

We leave you with some suggestions on how you can improve Air String if you are interested in creating a project that combines audio and video.

7.2 Future Propositions

Add a hardware distance sensor that detects the distance between the camcorder and fingers, and make the distance affect the volume level of a synthesized string sound.

Detect skin colors and keep track of the movements of fingers, so that a player does not have to wear a marker cap or tape on her/his fingers to play.

Expand the range of notes, rearrange notes and add chords so that multiple notes can be played simultaneously.

.

25

CHAPTER 8: REFERENCES

[1] Internet sources: various websites on Karplus Strong algorithm, musical note frequency, RGB color chart and more.

[2] Xilinx UG230 Spartan-3E Starter Kit Board User Guide.

www.bo.infn.it/~falchier/teaching/ug230.pdf

[3] Image Processing Toolbox.

http://www.mathworks.in/products/image/

[4] Steve on Image Processing.

http:// blogs.mathworks.com/steve/

26

APPENDIX A: LISING OF CODE FILES

AIR STRING

Air_string.m

Findgreen.m

Findgreenblock.m

Initialization.m

Karplus.m

Pluck.m

String.bmp

AIR PONG

Air_pong.m

Findgreen.m

Findgreenblock.m

Initialization.m

Pong.bmp

Pong.v

Pong.ucf

27

28