A Project Report
On
Air String And Air Pong
Submitted in the Partial Fulfillment of the
Requirement for the award of
Bachelor of Technology
in
Electronics & Communication Engineering
By
Ramesh Agarwal (20085094)
Manish Singh (20085027)
Ravi Pratap Gond (20085061)
Ashwarya Pratap Singh (20075087)
Under the guidance of
ASIM MUKHERJEE
Assistant Professor
Department of Electronics & Communication Engineering
Motilal Nehru National Institute of Technology Allahabad
Allahabad – INDIA
0
Department of Electronics & Communication Engineering Motilal
Nehru National Institute of Technology Allahabad
Allahabad – INDIA
CERTIFICATE
This is to certify that the work contained in the thesis titled “Air String And Air
Pong” submitted by Ramesh Agarwal, Manish Singh ,Ravi Pratap Gond and
Ashwarya Pratap Singh in the partial fulfillment of the requirement for the award of
Bachelor of Technology in Electronics and Communication Engineering to the
Electronics and Communication Engineering Department, Motilal Nehru National
Institute of Technology, Allahabad, is a bonafide work of the students carried out
under my supervision.
Date:
Place:
Mr . Asim mukherjee
Asst.professor
ECE Department
MNNIT, Allahabad
i
ACKNOWLEDGEMENT
It is a great privilege for us to express our deep sense of gratitude to our supervisor
Mr.Aseem Mukherjee of Electronics and Communication Engineering Department,
Motilal Nehru National Institute of Technology Allahabad for his stimulated guidance
and profound assistance. We shall always cherish our association with him for his
constant encouragement and freedom to thought and action that herendered to us
throughout our term paper project. We also feel a great pleasure to thank one and all
that helped us in carrying out this project.
Date:
Place:
Ramesh Agarwal (20085094)
Manish Singh (20085027)
Ravi Pratap Gond (20085061)
Ashwarya Pratap Singh (20075087)
ii
ABSTRACT
Our project is Air String, a synthesized string instrument that can be played in real
time by waving fingers with bright green color tips in motion of stroking a string in
front of a camcorder. Our implementation is based on Karplus Strong algorithm.
We got the idea for the product Microsoft Kinect of Xbox 360. The original idea
was to implement a synthesized string instrument that can be played in the air similar
to Air Guitar.The concept of the project is to provide a user interface similar to that of
playing the harp except for the fact that there is no physical instrument in front of the
user. Instead, the user’s finger motion in the air is recorded in real time to play virtual
strings of different notes. We liked the idea of combining the visual component
(VGA) and the audio component (audio codec) together.
iii
SUMMARY
For the project, we used Spartan 3E Platform FPGA Startar Kit,a XST 8.0 MP
Webcam,a HCL VGA compatible monitor at a 640x480 resolution (connected to the
Spartan 3E board via VGA Video Port) and a Stereo Speaker. We keep track of the
movement of a player’s fingers to decide whether to play a string, which musical note
of a string to play and whether to play the same note for the second time. We can
detect the movements of fingers by detecting the changes in RGB values of pixels on
the screen. For that, a player should put bright green color marker caps on her/his
finger tips (or wrap the fingers with color tapes). The monitor and the webcam face
the player while the webcam shoots a video of the player’s finger movements. The
monitor screen shows where the fingers are without a mirror effect along with white
lines and letters in the background. The lines indicate the each section for different
notes and the letters tell the player which note the section is allocated to.
iv
TABLE OF CONTENT
Page No.
CERTIFICATE i
ACKNOWLEDGEMENT ii
ABSTRACT iii
SUMMARY iv
CHAPTER 1: INTRODUCTION
1.1 Introduction and Task Analysis 1
CHAPTER 2: REVIEW OF ANALOGOUS CIRCUIT 4
CHAPTER 3:DESIGN AND IMPLEMENTATION 6
3.1 Hardware 6
3.2 Embedded Web Server Protocol Analysis 7
3.3 TCP/IP Stack 8
3.4 HTTP Protocol 14
CHAPTER 4:COMPOSITION OF ELECTRICAL CIRCUIT 17
4.1 Microcontroller 17
4.1.1 Pin Description 18
4.1.2 I/O Ports 21
4.2 ENC28J60 Ethernet Controller 22
4.2.1 Overview 22
4.2.2 Features 23
4.2.3 Pin Configuration 24
CHAPTER 5: MODEL INVESTIGATION 25
CHAPTER 6: CONCLUSION 29
CHAPTER 7: REFERENCES 30
v
APPENDIX A : LISTING OF CODE FILES 31
APPENDIX B : PROTEUS ISIS SCHEMATIC 32
APPPENDIX C: COST 3
LIST OF FIGURES
SERIAL NO FIGURE NAME PAGE NO
vi
2.1.1 KARPLUS ALGORITHM CHART 3
2.1.2 GRAPH BETWEEN VOLTAGE AND INPUT 4
2.2.1 FREQUENCIES OF MUSICAL NOTES 5
3.1.1 VGA MONITOR FOR AIR PONG 6
3.1.2 CIRCUIT DIAGRAM FOR VGA 6
3.1.3 CONNECTION INSIDE VGA 7
3.1.4 VGA SIGNAL TIMING 7
3.2.1 VGA TIMING WAVEFORMS 9
4.4.1 DIFFERENT NOTES OF GUITAR CORDS 16
5.1.1 RAY DIAGRAM FOR SSSM 18
5.2.1 STSM 20
5.3.1 FREQUENCY REGULATION CHART 21
5.3.2 FREQUENCY REGULATION WAVEFORM 22
vii
viii
CHAPTER 1
1.1 INTRODUCTION AND TASK ANALYSIS
Our project is Air String, a synthesized string instrument that can be played in
real time by waving fingers with bright green color tips in motion of stroking a string
in front of a camcorder. Our implementation is based on Karplus Strong algorithm.
We got the idea for the product Microsoft Kinect of Xbox 360. The original
idea was to implement a synthesized string instrument that can be played in the air
similar to Air Guitar.The concept of the project is to provide a user interface similar
to that of playing the harp except for the fact that there is no physical instrument in
front of the user. Instead, the user’s finger motion in the air is recorded in real time to
play virtual strings of different notes. We liked the idea of combining the visual
component (VGA) and the audio component (audio codec) together.
For the project, we used Spartan 3E Platform FPGA Startar Kit,a XST 8.0 MP
Webcam,a HCL VGA compatible monitor at a 640x480 resolution (connected to the
Spartan 3E board via VGA Video Port) and a Stereo Speaker. We keep track of the
movement of a player’s fingers to decide whether to play a string, which musical note
of a string to play and whether to play the same note for the second time. We can
detect the movements of fingers by detecting the changes in RGB values of pixels on
the screen. For that, a player should put bright green color marker caps on her/his
finger tips (or wrap the fingers with color tapes). The monitor and the webcam face
the player while the webcam shoots a video of the player’s finger movements. The
monitor screen shows where the fingers are without a mirror effect along with white
lines and letters in the background. The lines indicate the each section for different
notes and the letters tell the player which note the section is allocated to.
For prospective players of our virtual instrument, now we explain how
to interact with our program and play a song. Our program is set to look for a bright
green color as a virtual stroker (plucker) of a string. Wear a green marker cap or tape
on your finger. Face the monitor and the webcam, so you can see where your fingers
are and which sections on the screen to aim to play a note of your choice. For
example, to play middle C, aim for and cover the section labeled middle C on the
screen with the green marker cap or tape. If you want to play the same note for the
second time consecutively, you need to uncover the section by moving the finger
away from it and place the finger in the section again covering it with green. To play
a note, you can waive your fingers back and forth or move them horizontally to cover
2
CHAPTER 2: High Level Design
2.1Karplus Strong Algorithm
We employed Karplus Strong algorithm to implement a string. This algorithm is
surprisingly simple yet works very well. For a piano, two or three strings are used per
note. Since our user interface (plucking or stroking rather horizontally) is quite
different from that for a piano (striking down vertically), we construct one string per
note and the sound we synthesizes is closer to that of a guitar than to that of a piano.
Wikipedia definition of Karplus Strong string synthesis is a method of physical
modeling synthesis that loops a short waveform through a filtered delay line to
simulate the sound of a hammered or plucked string or some types of percussion.
The actual implementation of the algorithm for our project is depicted in the diagram
below:
FIGURE NO- 2.1.1
The hardware components to implement a string consist of a shift register, a phase
shifter and a low pass filter. The basic concept here is that an input pulse goes through
a certain length of shift register for a coarse tuning and it goes through a phase shifter
for a fine tuning. Then the output from the phase shifter goes through a simple low
pass filter which adds a delayed version (previous output) to the output and divides
3
the total by 2. The simple low pass filter basically averages two values before the
result is fed back into the shift register. For an input pulse, we chose a saw tooth wave
with a sharp raise at the beginning. This works very well for a nice string sound and it
works much better than some white noise. The amplitude of the pulse is 1, and the
step values of the pulse to be fed into the shift register are converted to a 3.17 number
format that is used in our hardware design. For example, 0.5 in 3.17 format is
represented as 0.5*2^17 = 20.65536. For your reference, mathematical representation
of the phase shifter is y(n) = {x(n) - y(n-1)} * η + x(n-1).
FIGURE NO-2.1.2
4
2.2 Frequencies of musical notes that we implemented are
tabulated below:
.
FIGURE NO-2.2.1
The length of the shift register and the value of η are related to the pitch (frequency)
of a note. Our default sampling rate, fs, is 8,000Hz. The length of a shift register, N,
for a note of frequency, fo, can be obtained from the equation fs/fo = N . For middle C,
the length of shift register that we need is then 8000/261.626 = 30.58. We set N equal
to 30. The sample delay, Δ, for middle C is then 0.58 and is defined in terms of η as
Δ=(1-η)/(1+η). Conversely, η=(1-Δ)/(1+Δ). The low pass filter uses a sign extended
right shift for damping (decay factor). 0.5 is the maximum value for damping.
However, we slightly lowered the decay factor to 0.4921875 because this produced
much better string sounds than when damping was 0.5.
5
CHAPTER 3: VGA__________________________________________________________________________________
3.1 Driving a VGA monitor for Air Pong
A VGA monitor requires 5 signals to display a picture:
R, G and B (red, green and blue signals).
HS and VS (horizontal and vertical synchronization).
FIGURE NO-3.1.1
The R, G and B are analog signals, while HS and VS are digital signals.
The Spartan®-3E FPGA Starter Kit board includes a VGA display port via a DB15
connector. Connect this port directly to most PC monitors or flat-panel LCDs using a
standard monitor cable. As shown in Figure 6-1, the VGA connector is the left-most
connector along the top of the board.
FIGURE NO-3.1.2
6
FIGURE NO-3.1.3
The Spartan-3E FPGA directly drives the five VGA signals via resistors. Each color
line has a series resistor, with one bit each fo r VGA_RED, VGA_GREEN, and
VGA_BLUE. The series resistor, in combination with the 75Ω termination built into
the VGA cable, ensures that the color signals remain in the VGA-specified 0V to
0.7V range. The VGA_HSYNC and VGA_VSYNC signals using LVTTL or
LVCMOS33 I/O standard drive levels. Drive the VGA_RED, VGA_GREEN, and
VGA_BLUE signal s High or Low to generate the eight colors shown.
FIGURE NO-3.1.4
VGA signal timing is specified, published, co pyrighted, and sold by the Video
Electronics Standards Association (VESA). The following VGA system and timing
7
information is provided how the FPGA might drive VGA monitor in 640 by 480
mode.
3.2 Signal Timing for a 60 Hz, 640x480 VGA Display
CRT-based VGA displays use amplitude-modulated, moving electron beams (or
cathode rays) to display information on a phosphor-coa ted screen. LCDs use an array
of switches that can impose a voltage across a small amount of liquid crystal, thereby
changing light permittivity through the crystal on a pixel-by-pixel basis. Although the
following description is limited to CRT displays, LCDs have evolved to use the same
signal timings as CRT displays. Consequently, the following discussion pertains to
both CRTs and LCDs.
Within a CRT display, current waveforms pass through the coils to produce magnetic
fields that deflect electron beams to tr ansverse the display surface in a raster pattern,
horizontally from left to right and vertically from top to bottom. As shown in Figure ,
information is only displayed when the beam is moving in the forward direction—left
to right and top to bottom—and not during the time the beam returns back to the left
or top edge of the display. Much of the potential display time is therefore lost in
blanking periods when the beam is reset and stabilized to begin a new horizontal or
vertical display pass.
8
FIGURE NO-3.2.1
The display resolution defines the size of the beams, the frequency at which the beam traces across the display, and the frequency at which the electron beam is modulated.
Modern VGA displays support multiple display resolutions, and the VGA controller dictates the resolution by producing timing signals to control the raster patterns. The controller produces TTL-level synchronizing pulses that set the frequency at which current flows through the deflection coils, and it ensures that pixel or video data is applied to the electron guns at the correct time.
9
3.3 VGA Signal Timing
The signal timings in Table are derived for a 640-pixel by 480-row display using a 25
MHz pixel clock and 60 Hz ± 1 refresh. Figure 6-3 shows the relation between each
of the timing symbols. The timing for the sync pulse width (TPW) and front and back
porch intervals (TFPand TBP) are based on observations from various VGA displays.
The front and back porch intervals are the pre- and post-sync pulse times. Information
cannot be displayed during these times.
FIGURE NO-3.3.1
10
3.4 Frequency generator
A monitor always displays a picture line-by-line, from top-to-bottom. Each line is drawn from left-to-right.That's hard-coded, you cannot change that.
But you specify when the drawing starts by sending short pulses on HS and VS at fixed intervals. HS makes a new line to start drawing; while VS tells that the bottom has been reached (makes the monitor go back up to the top line).
For the standard 640x480 VGA video signal, the frequencies of the pulses should be:
Vertical Freq (VS) Horizontal Freq (HS)
60 Hz (=60 pulses per second) 31.5 kHz (=31500 pulses per second)
To create a standard video signal, there is more details to take care of, like the duration of the pulses and the relationship between HS and VS.
3.5 Our Video generator
Nowadays, VGA monitors are multisync, so can accommodate non-standard frequencies - no need to generate exactly 60Hz and 31.5KHz anymore (but if you are using an old (non-multisync) VGA monitor, you'll need to generate the exact frequencies).
Let's start with X and Y counter.
CounterX counts 768 values (from 0 to 767) and CounterY counts 512 values (0 to 511).
Now, CounterX is used to generate HS, and CounterY to generate VS. Using a 25MHz clock, we get 32.5KHz for HS and 63.5Hz for VS. The pulses need to be active long enough for the monitor to detect them. Let's use a 16 clocks pulse (0.64µs) for HS and a full horizontal line length pulse for VS (768 clocks or 30µs). That's shorter than what the VGA spec calls for but works fine anyway.
11
We generate the HS and VS pulses from D-flipflop (to get glitch free outputs).
The VGA outputs need to be negative, so we invert the signals.
Finally we can drive the R, G and B signals. As a first cut, we can use some bits of the X and Y counters to get nice square color patterns.
. and we get a picture on the VGA monitor!
CHAPTER 4: USING THE VGA__________________________________________________________________________________
12
4.1 Drawing a useful picture
The sync generator is best rewritten to be used as an HDL module where we generate R, G and B outside. Also the X and Y counters are more useful if they start counting from the drawing area.
Now we can use it to draw a border around the screen.
13
4.2 Drawing a paddle
Let's use a mouse to move the paddle left and right on the screen.
Now that "PaddlePosition" value is known, we can display the paddle.
4.3 Drawing the ball
The ball needs to move around the screen, and bounce back when it touches an object (border or paddle).
First we display the ball. It is a square 16x16 pixels. We activate the drawing of the ball when CounterX and CounterY reach its coordinates.
Now for the collisions. That's the difficult part of this project.
14
We could check the coordinate of the ball against each object on the screen and determine if there is a collision. But that would become quickly a nightmare as the number of objects increases.
Instead we define 4 "hot-spots" pixels, one in the middle of each side of the ball. If an object (border or paddle) redraws itself at the same time that the ball draws one of its "hot-spot", we know that there is collision on that side of the ball.
And finally we can draw all that together.
15
4.4 Screen Arrangements for Air String
For a user-friendly interface, we printed on the monitor screen white lines that outline sections and letters that correspond to notes of the sections. To create this background image that is printed on the monitor all the time while a player is playing, we used Paint program and created a black and white .bmp file as below:
FIGURE NO-4.4.1
Now the background image is stored in the memory and we have two images to send
to the monitor screen: one from the camcorder and one in the memory. We checked
the color value in the memory and if it was 0, black, then we chose the RGB value
from the camcorder for a pixel, otherwise we chose the one bit value from the
memory because the pixel was of white lines and letters. (A simple mux does the
trick.)
16
CHAPTER 5 :Hardware Design
5.1 String Synthesizer state machine (SSSM)
This state machine consists of a shift register, and the phase shifter and the low pass filter are implemented in combinatory logic.
To generate 16 notes of different frequencies, we set the default sampling rate at 8,000 Hz, which is the clock rate for all the state machines. Thus each state machine has a shift register of a different length and a different sample delay value for each phase shifter. A shift register consists of many 20-bit registers. (For better accuracy, we used 3.17 format instead of 3.13 format.) We chose not to use M4K blocks and it worked out well for us because 1) it turned out that we could not afford to add one more clock cycle to access the memory in the string synthesizer state machine because the string trigger state machine (STSM) at a much faster clock (VGA_CLK at 27 MHz, 3375 times faster) has to wait for the SSSM to send a signal before it can move onto the next state (this signaling between two state machines is explained later in details) and 2) we used 70% of M4K blocks to store the background image later. Overall we used about 50% of logic elements and resources available on the board to implement the entire system.
Basically, SSSM works as follows:
if (the string is plucked by STSM){ initialize the shift register with a saw tooth input pulse; send signal to STSM that the string is plucked (set a flag to 1); go to state 0; }else{ state 0: shift register values in the shift register (one right shift); send signal to STSM that the string is being played (set a flag to 0); go to state 1; state 1: update values for combinatory logic; go to state 0; }
State machine diagram is shown below:
17
FIGURE NO -5.1.1
5.2 String Trigger state machine (STSM)
This state machine checks for the presence of bright green color in a particular section for a corresponding note by counting a number of pixels, RGB value of which is specified as G > 10'h99 AND R < 10'h80 AND B < 10'h80. Each section for a note has 100x60=6,000 pixels and if there are more than 1,500 pixels whose RGB values meet the above requirement, then the state machine triggers an SSSM that plays a corresponding note.
STSM and SSSM signal each other (handshaking) so that they can coordinate their executions although they work under different clock rates. This way, when a user places a green marker cap in a section for a note on the screen and does not remove it
18
for a while, the string plays only one time and rests until the cap is removed from the section and placed there again for the second consecutive stoke.
Basically, the way STSM works is as follows:
state 0: if (VGA x and y coordinates are pointing to pixels in the section for this STSM) { count the number of bright green pixels; go to state 0; } else if (VGA x and y coordinates reached the end of screen) go to state 1; else go to state 0;
state 1: if (more than 1,500 bright green pixels are present) { if (the string has not been plucked) { signal the SSSM to feed in an input pulse to the shift register; set a flag to 1 to remember that the string has been plucked once; go to state 2; } else // there has been no change in the movement of a player since the last pluck { do not signal the SSSM to feed in an input pulse to the shift register; set a flag to 1 to remember that the string has been plucked once; go to state 4; } } else { do not signal the SSSM to feed in an input pulse to the shift register; set a flag to 0 to remember that the string has not been plucked; // the string is ready for the next pluck; go to state 4; }
state 2: if (SSSM signaled that the shift register will be initialized with an input pulse) go to state 3; else // wait for SSSM to catch up and signal that the string is read to be played. go to state 2;
state 3: do not signal the SSSM again to feed in an input pulse to the shift register; if (SSSM signals that the shift register is done being initialized)
19
go to state 4; else // SSSM is not done initializing the shift register go to state 3;
state 4: // reset registers for the next VGA screen check remember whether the string was plucked this time; set the green color detection counter to 0; go to state 5;
state 5: if (VGA x and y coordinates reached the end of the screen) go to state 0; else // green color detection always starts from the beginning of the screen. go to state 5;
Waiting for the SSSM which operates under a slower clock rate to send signals before it could go to the next state, STSM stays in a few wait states for many numbers of VGA screen sweeps (refreshes). It means that we skip many frames without checking for changes in RGB values of a section. However, this does not affect the accuracy of our program because any fast human hand movement is much slower than the VGA refresh rate and the most human eyes cannot discern discontinuity when the frame rate is over 100 FRS (frame rate per second).
State machine diagram is shown below:
FIGURE NO-5.2.1
20
5.3 Frequency regulatorWe needed to bring down AUD_DACLRCK (48 KHz) to 8,000 Hz sampling rate to
clock SSSMs. This frequency regulator simply uses a counter to wait and to generate
a lower frequency clock signal than the input clock signal. Using this frequency
regulator and muxes, we produced three different sampling frequencies, fs, 12 KHz,
8,000 Hz and 4,800 Hz respectively, and a player can choose three different sets of
notes using dip switches on the board. When all the SWs are set to 0, fs is the default
value of 8,000 Hz. If SW[0] = 1, fs is 4,800 Hz. If SW[17] = 1, fs is 12 KHz.
Although they are not separated by one octave exactly, they are all harmonics. This is
caused by the fact that we keep the same value for N in the equation, f s/fo = N, and
vary the value of fs, so the output frequency, fo, is not scalable by a player. However,
a player can enjoy different timbers of a string sound. The background image of white
lines and letters for notes does not change as a different sampling rate from the
default is chosen. Three sets of notes that are synthesized are tabularized below:
FIGURE NO-5.3.1
21
The figure below shows up to what number the counter should count to generate
clock of which frequency.
FIGURE NO-5.3.2
5.4 Screen FlipperWhen we display a video stream from a camcorder on a monitor, a mirror effect is
observed. This confuses a player when s/he faces the camcorder and the monitor to
aim for a certain section on the screen to play a note because as s/he moved her/his
hand from right to left, s/he sees the hand move from left to right. For a user friendly
interface, we corrected the mirror effect by adding Mirror_col module. Like Altera,
terasIC provides tutorial documents and demonstration project files. One of the
project files with a top module named DE2_CCD.v comes with Mirror_col module
that reverses the frame captured from the sensor in TRDB_DC2 camera. We modified
the Mirror_col and added to our project to reverse the screen. Because Mirror_col
22
uses two stack RAM as buffers, the left half side of the screen is actually one frame
behind the right half side of the screen. However, due to the fast VGA clock, this does
not affect the speed of visual (hand movement) that a player sees or audio (string
sound) that a player hears.
CHAPTER 6 :RESULTS__________________________________________________________________________________
6.1 Results of the Design
The Air String has a user friendly interface, and it makes it easy for anyone to play a song. The way it’s played is quite intuitive to most people without any skills in musical instruments. Although the letters for notes are printed on the screen for a quick reference, if a player memorizes the sequence of notes of a song and practices, which most musicians do, s/he can play the song much smoothly, fast and easily.
Unfortunately we did not have more time to add more features to Air String. A user can choose from three different timbers of a string sound, but the notes in each set are not exactly distanced by an octave. Also, the notes with a sampling rate different from the default do not match the corresponding letter on the screen because we have only one background image stored in memory. Using additional memory such as SRAM and re-designing the frequency generator could improve Air String.
A tip for a player: Having a bright yellow light shining over the camcorder from the behind helps the color of the marker cap to be detected as a bright green color easily. It helps the player play better.
To detect the change in the finger movements of a player, we count the number of pixels of a specific color (in our case, a bright green color whose RGB value in hex is 24h009900). To look for this color, we check if an RGB value from the camcorder satisfies the condition (VGA_G > 10'h99 && VGA_R < 10'h80 && VGA_B < 10'h80). We picked a green color of 24h009900 because we usually do not see this color around us so that it could be easily distinguished from any background. Instead of using absolute values for RGB for a color check, when we tried the relevant condition (mGreen > mRed && mGreen > mBlue), the color detection did not work very well thus making it harder for a player to play.
6.2 Trade-offThe way we designed the Air String is that any notes should not keep playing once it was played while a green marker cap stayed in the same section for long. We consider a player meant to play a string when of a section corresponding to a note is filled with bright green color pixels (1,500 pixels out of 6,000 pixels for a section). The number 1,500 seems to work the best for us when there is yellow light over the camcorder. If we lower the number, then a player does not need to place the entire
23
marker cap in a section and it could help increase the speed of finger movement from a note to another. However, this also more easily triggers a string, so sometimes a note plays more than once when a player did not mean to play the note twice consecutively. If we increase the number, then it takes a player a little more time to move from one note to anther away from it although this decrease the chance of a note being played more than once when a player did not mean to.
6.3 Accuracy in terms of timing issuesReversing the screen to remove the mirror effect for a player’s convenience was
tricky because we already had some issue with time, which was cause by signaling
(handshaking) between two state machines of two different clock rates as one state
machine with a faster clock has to wait for the other for many clock cycles while
VGA does not stop refreshing the screen. Air String needs to be played in real time
and there should not be any delay between a player’s finger movement and the video
stream on the monitor screen. Using stack RAMs as buffers to reverse the screen
causes VGA to display the left half side of the screen one frame late. If you have a
really good eye, then you might be able to detect it. However, it did not affect any of
our test players performance.
24
CHAPTER 7: Conclusion and Future Work__________________________________________________________________________________
7.1 ConclusionsOverall, the project was a success. We produced a working model with a user friendly interface. We believe Air String is easy for anyone to learn to play and quite entertaining and educational for children to stimulate their interest in musical instruments. Throughout the project, we had an opportunity to demonstrate our skills that we obtained from previous lab assignments and also had an opportunity to learn much more about FPGA and new things such as TV decoder and Karplus Strong algorithm.
We leave you with some suggestions on how you can improve Air String if you are interested in creating a project that combines audio and video.
7.2 Future Propositions
Add a hardware distance sensor that detects the distance between the camcorder and fingers, and make the distance affect the volume level of a synthesized string sound.
Detect skin colors and keep track of the movements of fingers, so that a player does not have to wear a marker cap or tape on her/his fingers to play.
Expand the range of notes, rearrange notes and add chords so that multiple notes can be played simultaneously.
.
25
CHAPTER 8: REFERENCES
[1] Internet sources: various websites on Karplus Strong algorithm, musical note frequency, RGB color chart and more.
[2] Xilinx UG230 Spartan-3E Starter Kit Board User Guide.
www.bo.infn.it/~falchier/teaching/ug230.pdf
[3] Image Processing Toolbox.
http://www.mathworks.in/products/image/
[4] Steve on Image Processing.
http:// blogs.mathworks.com/steve/
26
APPENDIX A: LISING OF CODE FILES
AIR STRING
Air_string.m
Findgreen.m
Findgreenblock.m
Initialization.m
Karplus.m
Pluck.m
String.bmp
AIR PONG
Air_pong.m
Findgreen.m
Findgreenblock.m
Initialization.m
Pong.bmp
Pong.v
Pong.ucf
27
28