hen computers were first introduced, they were large and expensive and
O bjec t i vesIn this tutorial, you will learn to:■ Download the components
necessary to run Microsoft Agent.
■ Enhance Windows applications using multimedia.
■ Create applications that interact with users.
■ Use Microsoft Agent in a C# application.
O ut l in e27.1 Microsoft Agent27.2 Downloading Microsoft
Agent Components27.3 Test-Driving the Phone Book
Application27.4 Constructing the Phone
Book Application27.5 Wrap-Up
were used primarily to perform arithmetic calculations. Multimediaapplications, which use a variety of media, including graphics, animation,
video and sound, were made impractical by the high cost and slow speed of com-puters. However, today’s affordable, ultrafast processors are making multimedia-based applications commonplace. As the market for multimedia explodes, usersare purchasing computers with faster processors, larger amounts of memory andwider communications bandwidths needed to support multimedia applications.
Users are seeing exciting new three-dimensional multimedia applicationsthat interact with the user by means of animation, audio and video. Multimediaprogramming is an entertaining and innovative field, but one that presents manychallenges. C# enables you to include such multimedia presentations in yourapplications.
In this tutorial, you will explore the Microsoft Agent technology, which usesentertaining, animated three-dimensional cartoon characters to interact with theapplication user. You will create a Phone Book application that uses one of thepredefined Agent characters.
27.1 Microsoft AgentIn this tutorial, you will create a Phone Book application that displays people’sphone numbers, using Microsoft Agent to interact with users and enhance theapplication. This application must meet the following requirements:
Application Requirements
A software company’s customer service department is responsible for call-ing clients. They need a quick way to access their clients’ phone numbersand have asked you to develop an application that stores and retrievesthe names and numbers of their clients. The service-department employ-ees want an application that employs multimedia (using the MicrosoftAgent character, Peedy the Parrot) to allow them to retrieve the phonenumbers by speaking the clients’ names and also by selecting clients’names with the mouse.
W
723hts Reserved.
724 Introducing Multimedia Using Microsoft Agent Tutorial 27
Microsoft Agent is a technology used to add interactive animated characters toWindows applications or Web pages. Microsoft Agent characters can speak (byusing voice synthesis) and respond to user input (by using speech recognition).Microsoft employs its Agent technology in such applications as Word, Excel andPowerPoint, where they help users understand how to use the application.
The Microsoft Agent control provides you with access to four predefined char-acters—Genie (a genie), Merlin (a wizard), Peedy (a parrot) and Robby (a robot).Each character contains unique animations that you can use in their applications toillustrate different instructions and actions. For instance, the Peedy character-ani-mation set includes several flying animations which you can use to move Peedyacross the screen. Microsoft provides basic information on Agent technology at:
www.microsoft.com/msagent/default.asp
Microsoft Agent technology enables users to interact with applications andWeb pages by using speech. When the user speaks into a microphone, the controluses a speech-recognition engine, an application that translates vocal sound inputfrom a microphone into a language that the computer understands. The MicrosoftAgent control also uses a text-to-speech engine, which allows the Microsoft Agentcharacters to speak lines of text. A text-to-speech engine is an application thattranslates typed words into sound, which users hear through headphones or speak-ers connected to a computer. Microsoft provides speech-recognition and text-to-speech engines for several languages at
SELF-REVIEW 1. A translates typed words into sound.
2. The application that translates vocal sound input from a microphone to a language under-stood by the computer is called the .
Answers: 1) b. 2) a.
27.2 Downloading Microsoft Agent ComponentsMicrosoft Agent characters can be used as visual aids for applications. TheseAgents also allow users to speak to, listen to and interact with the characters. Thistutorial demonstrates how to use the Microsoft Agent characters to build thePhone Book application. To run this tutorial’s application, you must download andinstall the Agent control, speech-recognition engine, text-to-speech engine andPeedy character definition from the Microsoft Agent Web site. Begin by visiting:
This page (Fig. 27.1) displays a list of Microsoft Agent downloads. The first compo-nent you need to download is the Agent character file. Click the Microsoft Agentcharacter files link (Fig. 27.1).
Clicking this link directs users to the location where the Microsoft Agent char-acter files can be downloaded (Fig. 27.2). Select the Peedy character from the drop-down list and click the Download selected character link. Save the file Peedy.exeto your computer, and install the Peedy character files by double clicking this file.[Note: You will need to download and install the other characters as well to com-plete this tutorial’s exercises.]
a) speech-recognition engine b) text-to-speech enginec) character-animation set d) All of the above.
a) speech-recognition engine b) text-to-speech enginec) character-animation set d) All of the above.
Figure 27.1 Microsoft Web page containing Agent-related downloads.
Figure 27.2 Location for downloading the Microsoft Agent character.
Next, you should download the text-to-speech engine. Return to the MicrosoftAgent Downloads Contents list by scrolling to the top of the current page, andclick the Text-to-speech engines link. Click the ComboBox to display the drop-down list, and select the engine that supports American English. Click the Down-load selected engine link to save the file tv_enua.exe. Double click this file onceit has been downloaded to install the text-to-speech engine. You may need torestart your machine after this installation.
You must also download the speech-recognition engine. Return to theMicrosoft Agent Downloads Contents list by scrolling to the top of the currentpage, and click the Speech recognition engines link. Click the Download theMicrosoft Speech Recognition Engine link to save the file actcnc.exe. Doubleclick this file once it has been downloaded to install the speech-recognition engine.The installation process will walk you through the configuration of your micro-phone. Once you have downloaded and installed all of the components, you areready to use Microsoft Agent.
[Note: Windows XP users may need to download an additional file for speechcomponents to function correctly. If you find in the next box that your agents do notspeak, return to the Microsoft Agent Downloads Contents list by scrolling to thetop of the current page, and click the SAPI runtime support link. Click the Down-
726 Introducing Multimedia Using Microsoft Agent Tutorial 27
load the Microsoft SAPI 4.0a runtime binaries link to save the file spchapi.exe.Double click this file once it has been downloaded to install support binaries.]
27.3 Test-Driving the Phone Book ApplicationRecall that you will be creating the Phone Book application to allow users tosearch for a phone number using an interactive Microsoft Agent character. YourPhone Book application, which you build in the next section, will contain people’snames and phone numbers for the customer-service department. You begin by test-driving the application. Then, you will learn the additional C# technologies that youwill need to create your own version of this application.
Test-Driving the PhoneBook Application
1. Opening the completed application. Open the C:\Examples\Tutorial27\CompletedApplication\PhoneBook directory to locate the Phone Bookapplication. Double click PhoneBook.sln to open the application in VisualStudio .NET.
2. Running the Phone Book application. Select Debug > Start to run theapplication (Fig. 27.3).
Figure 27.3 Phone Book application Form.
3. Calling Peedy. Click the Call Peedy Button to display Peedy. Peedy fliesonto the screen into a position beneath the Form, waves and speaks theinstructions shown in Fig. 27.4. When he is finished speaking, Peedy goesinto a resting position (Fig. 27.5).
Figure 27.4 Peedy appears after the Call Peedy Button is clicked.
4. Using the ComboBox to select a name. After Peedy appears on the screen,the ComboBox is enabled. Select the name Howard from the list (Fig. 27.6).Notice that the Call Peedy Button is disabled, as Peedy is already on thescreen. After Howard is selected, Peedy executes several animations. He firstappears to be thinking (Fig. 27.7), then tells you Howard’s number(Fig. 27.8), then smiles for the user (Fig. 27.9).
Figure 27.9 Peedy smiles after speaking the phone number.
5. Providing a voice command. Press the Scroll Lock key. A box appearsbeneath Peedy (called a status box) that displays information about Peedy’sactions. The status box states that Peedy is listening for your command, as inFig. 27.10. Speak the name John into your microphone.
Figure 27.10 Peedy listening for a voice command.
If Peedy hears and understands your command, the status box displaysthe command he heard, as in Fig. 27.11. Otherwise, you must repeat the com-mand clearly, so Peedy can understand.
728 Introducing Multimedia Using Microsoft Agent Tutorial 27
SELF-REVIEW 1. The displays information about the Microsoft Agent character’s actions.
2. You must press the key for Peedy to listen to your voice commands.
Answers: 1) b. 2) c.
27.4 Constructing the Phone Book ApplicationNow that you have test-driven the Phone Book application, you need to analyzethe application, using pseudocode. The Microsoft Agent character Peedy helps
Figure 27.11 Peedy heard the voice command.
When Peedy successfully hears your command, he gestures as if he isthinking about your request, then displays John’s phone number (Fig. 27.12).You will have noticed that Peedy performs several gestures during this appli-cation. You will learn how to control these gestures later in the tutorial.
Figure 27.12 Peedy displays the requested phone number.
6. Viewing the context menu. Right click Peedy and notice that a contextmenu appears (Fig. 27.13). Selecting one of the names (John, Jennifer orHoward) will cause the same actions to occur as when a name was chosenfrom the ComboBox.
Figure 27.13 Context menu window.
7. Closing the application. Close your running application by clicking itsclose box.
8. Closing the IDE. Close Visual Studio .NET by clicking its close box.
(cont.)
Commands in pop-up window
a) balloon b) status boxc) help window d) text window
users search for a specific telephone number. The user can click the ComboBox toselect a name and retrieve the specified telephone numbers. However, thanks to theenhancement of the Microsoft Agent character, the user also can communicate ver-bally with Peedy. The user can retrieve a phone number simply by pressing theScroll Lock key and speaking the name of a person into a microphone connected tothe computer. Peedy listens for a name, and if he recognizes it, displays and speaksthe number to the user. The following pseudocode describes the basic operation ofthe Phone Book application.
When the Form loads:Display names in the ComboBoxLoad Peedy the Parrot character into the Agent controlObtain Peedy the Parrot from the Agent control’s Characters propertyAdd names as commands for Peedy the Parrot
When the user clicks the Call Peedy Button:Display Peedy the Parrot and have the parrot speak the instructionsEnable ComboBox containing people’s namesDisable the Call Peedy Button
When the user selects a name from the ComboBox:Have Peedy the Parrot speak the name and phone number of the person
selected by the user
When the user speaks a name to Peedy the Parrot:Have Peedy the Parrot speak the name and phone number of the person
selected by the user
When Peedy the Parrot hides:Disable ComboBox containing people’s namesEnable the Call Peedy Button
Now that you have test-driven the Phone Book application and studied itspseudocode representation, you will use an ACE table to help you convert thepseudocode to C#. Figure 27.14 lists the actions, controls and events that you willhelp you complete your own version of this application.
Action/Control/Event(ACE) Table for the
Phone BookApplication
Action Control/Object Event
Label the application’s controls lblInformation Application isrun
FrmPhoneBook Load
Display names in the ComboBox cboName, m_strNameList
Load Peedy the Parrot character into the Microsoft Agent control
objMainAgent
Obtain Peedy the Parrot from the Agent control’s Characters property
objMainAgent, m_objMSpeaker
Add names as commands for Peedy the Parrot
m_objMSpeaker, m_strNameList
btnCall Click
Display Peedy the Parrot and have the parrot speak the instructions
m_objMSpeaker
Enable ComboBox containing people’s names cboName
Disable the Call Peedy Button btnCall
Figure 27.14 ACE table for the Phone Book application. (Part 1 of 2.)
730 Introducing Multimedia Using Microsoft Agent Tutorial 27
Now that you understand the purpose of the Phone Book application, you willbegin to create it. In the following box, you will add the Microsoft Agent control toyour Toolbox.
Customizing the Toolboxfor the Phone Book
Application
Action Control/Object Event
cboName Selected-IndexChanged
Have Peedy the Parrot speak the name and phone number of the person selected by the user
Figure 27.14 ACE table for the Phone Book application. (Part 2 of 2.)
1. Copying the template to your working directory. Copy the C:\Examples\Tutorial27\TemplateApplication\PhoneBook directory to yourC:\SimplyCSP directory.
2. Opening the Phone Book application’s template file. Double click Phone-Book.sln in the PhoneBook directory to open the application in VisualStudio .NET.
3. Adding the Microsoft Agent to the Toolbox. Before you begin designing theForm, you must make the Microsoft Agent accessible. To do this, you mustadd the Microsoft Agent control to the Toolbox window. Select Tools >Add/Remove Items…, as in Fig. 27.15. The Customize Toolbox dialogappears (Fig. 27.16). Select the COM Components tab and search for theMicrosoft Agent Control 2.0 item. Then, click its CheckBox to select thecontrol as in Fig. 27.16. Click the OK Button. The Agent control nowappears in the Windows Forms group in the Toolbox, as in Fig. 27.17.
The visual design of the Form (with the exception of the Agent control) is pro-vided in the template application. The next step after adding the Microsoft Agentcontrol to the Toolbox is to place a Microsoft Agent control on the Form.
Figure 27.16 Customize Toolbox dialog.
Figure 27.17 Agent Control in the Toolbox.
4. Saving the project. Select File > Save All to save your modified code.
732 Introducing Multimedia Using Microsoft Agent Tutorial 27
Adding the MicrosoftAgent Control to the
Application
Now that you have placed the Agent control on the application’s Form, you willdefine the event handlers for the Phone Book application. These event handlerswill define how the Agent responds to user actions. You begin writing code in thefollowing box.
Using Code to Displaythe Peedy Agent
Character
1. Placing the Microsoft Agent control on the Form. To meet the applicationrequirements, you must add a Microsoft Agent control to your Form, whichwill be used to display and manage the actions of the Microsoft Agent charac-ters in your application. Drag and drop the Microsoft Agent control from theToolbox onto the Form. Change the Microsoft Agent control’s Name propertyfrom the default (axAgent1) to objMainAgent. Change the Microsoft Agentcontrol’s Location property to 16, 48 (Fig. 27.18). Because the control iconis not visible when the application runs, the location on the Form does notaffect the application’s appearance. However, it is good practice to place thecontrol so that it is not overlapping another control on the Form.
Figure 27.18 Form format of Phone Book.
Though you set the Agent control’s Location property in this step, youwill write code later that will determine where the Agent character appearswhen the application is running. As you saw in the test drive, the MicrosoftAgent character can be displayed outside the application’s Form.
2. Running the application. Select Debug > Start to run your application(Fig. 27.19). The Call Peedy Button will be used to make the Agent (in thiscase, Peedy) appear on the screen. However, clicking the Button does notcause any action to take place yet, because the Button’s Click event han-dler has not been defined. Notice that the ComboBox next to the Call PeedyButton is disabled. You will use code to enable and fill the ComboBox withnames shortly.
Figure 27.19 Phone Book Form with no functionality.
3. Closing the application. Close your running application by clicking its closebox.
Good ProgrammingPractice
Controls in a container should notoverlap. Place your controls so thatthey are clearly separated from oneanother, making the design of yourapplication a clean one.
1. Rearranging and commenting the new control declaration. Go to codeview. Move the declaration for the Microsoft Agent control from line 23 ofyour code to line 25 of Fig. 27.20, then add the comment in line 24.
Figure 27.20 Rearranging and commenting the new control declaration.
2. Declaring instance variables. Add lines 32–40 of Fig. 27.21 in your code todeclare the instance variables you will use in several event handlers. Inaddition to the Microsoft Agent objMainAgent object that manages all theapplication’s characters, you also need an object to represent the currentcharacter (sometimes referred to as the speaker). In this example, the cur-rent character will always be Peedy the Parrot. Line 32 declares a variableto represent this Agent character. The m_objMSpeaker variable is declaredas an AgentObjects.IAgentCtlCharacter object.
Figure 27.21 Declaring and creating arrays in the Phone Book application.
The m_strNameList and m_strNumberList arrays are then declared andfilled. The m_strNameList array (lines 35–36) stores the names of people inthe phone book (John, Jennifer and Howard), and the m_strNumberListarray (lines 39–40) stores the corresponding phone number for each person((555) 555-9876, (555) 555-1234 and (555) 555-4567).
3. Writing code that executes when the application loads. The next step increating the Phone Book application is to use the Form’s Load event to exe-cute code before the application becomes available to the user. Double clickthe Form in the Windows Form Designer to generate the Load event han-dler, and enter code view. Be sure to add the comment and break theheader as shown in Fig. 27.22 so that the line numbers in your code matchthose presented in this tutorial. Then, add lines 147–154 to the Load eventhandler.
734 Introducing Multimedia Using Microsoft Agent Tutorial 27
Figure 27.22 Adding items to the ComboBox.
The intCounter variable (line 148) is used to iterate through them_strNameList array in the for statement (lines 150–154). This statementfills the ComboBox with the contact names that are stored in them_strNameList array. This allows the user to select names by using theComboBox instead of speaking the name to Peedy.
4. Initializing the Peedy Agent character. Add lines 156–159 of Fig. 27.23 tothe event handler. Line 157 loads the Peedy character into objMainAgent.The first argument of the Load method is a string used to represent theAgent character being loaded. In this case, use the "Peedy" string. Thesecond argument is a string representing the file where the character isdefined ("Peedy.acs").
Figure 27.23 Loading the Microsoft Agent character.
Several characters can be loaded into a Microsoft Agent control. In thisexample, however, we will only be using Peedy the Parrot, so no more char-acters will be loaded. Line 159 assigns the loaded character tom_objMSpeaker. The character is accessed using the Characters propertyof our Microsoft Agent control. This property contains a collection of char-acters that have been loaded into the Microsoft Agent control.
The m_objMSpeaker variable can now be used to reference the PeedyAgent character. It is not necessary to create a separate object of the Agen-tObjects.IAgentCtlCharacter type—the Peedy character can beaccessed with the expression objMainAgent.Characters[ "Peedy" ]. Wehave created m_objMSpeaker to increase application clarity. If any othercharacters had been loaded into the Microsoft Agent control, they can beaccessed by replacing the string "Peedy" with the name of the character tobe accessed.
5. Inserting commands in the Agent Commands context menu. Add lines 161–169 of Fig. 27.24 to your event handler. This code uses the same counterfrom the previous step in its for statement (lines 162–169). The header ofthe for statement resets the counter variable’s value to 0. Then, the forstatement adds names from the m_strNameList to the Peedy character asvoice-enabled commands. The list of valid commands for a character isstored in the Commands property of the AgentObjects.IAgentCtlCharac-ter object.
Figure 27.24 Adding commands to Peedy’s context menu.
The Add method of the Commands property adds a new command to thecommand list. The Add method takes five arguments. The first three argu-ments are strings and the last two are bool values. The first argument iden-tifies the command name. This value enables access to the command fromyour application. The second argument is a string that appears in a contextmenu when the user right clicks Peedy. The third string represents theword(s) for which Peedy listens when users make a verbal request. Thefourth argument indicates whether the command is enabled. If so, the Agentcharacter will respond to the spoken command. The final argument specifieswhether the command is visible in the Commands context menu, which youhave already seen in Fig. 27.13.
In this example, you set the first three arguments to the same value in them_strNameList array and set the last two arguments to true for each name.Now Peedy understands users if they speak any of the names found in them_strNameList array—John, Jennifer or Howard.
A Command event is raised when the user selects the command from theCommands pop-up window or speaks the command into a microphone. Com-mand events are handled by the Command event handler of the MicrosoftAgent control (objMainAgent, in this example).
(cont.)
Good ProgrammingPractice
Use a variable as an alternative ref-erence to an object when the fullreference to the object is long. Thisimproves code readability.
736 Introducing Multimedia Using Microsoft Agent Tutorial 27
6. Defining the btnCall_Click event handler. After defining the Form’s Loadevent, you must create an event handler for the Call Peedy Button. Whenthis Button is clicked, Peedy should appear and interact with the user. Tocreate the Button’s Click event handler, double click the Call Peedy But-ton in the Windows Form Designer. Be sure to add the comments and breakthe header as shown in Fig. 27.25 so that the line numbers in your codematch those presented in this tutorial. Then, add lines 178–183 of Fig. 27.25to your code.
Figure 27.25 Displaying the Microsoft Agent character.
Line 178 uses the Show method to display Peedy on the screen. Specifying0 as the argument makes Peedy fly onto the screen and land at his defaultlocation (towards the top left of the screen). However, if 1 is passed to themethod, Peedy’s image pops onto the screen at his default location (withoutthe flying animation).
After the Show method is called, Peedy appears at (or flies to) the defaultposition on the screen. The command in lines 181–183 causes Peedy to moveto a new location. These lines use the MoveTo method of the m_objMSpeakerAgent object. The MoveTo method takes three arguments. The first two arean x-coordinate and a y-coordinate, both of the short type. The short typeis similar to the int type, but short variables occupy less space in memoryand therefore cannot hold larger int values. Convert.ToInt16 converts thevalue of Cursor.Position.X - 100 from an int value to a short value,where Cursor.Position.X contains the current position of the mousepointer. We have subtracted 100 to specify a position that is 100 pixels to theleft of the mouse pointer. The same was done for the y-coordinate. In thiscase, the method moves the Agent character 100 pixels to the left and 130pixels below the mouse pointer at the time the Call Peedy Button is clicked.This keeps Peedy near the application’s Form. The third argument of theMoveTo method is Peedy’s speed. This argument dictates the amount of time,in milliseconds, between frames in Peedy’s animation.
(cont.)
Move Peedy
Make Peedy fly onto the screen
GUI Design Tip
Locate the Microsoft Agent charac-ter near the application’s Form.
7. Coding Peedy’s greeting. Add lines 185–195 of Fig. 27.26 to your Click eventhandler. Line 185 uses the Play method to command Peedy to perform anaction. This method plays the character animation specified by the stringargument that is passed. In this line, Peedy’s Wave animation is played—hewaves hello to the user. [Note: We use only a few of the Peedy character’savailable animations. To see a listing of the available animations, please visitmsdn.microsoft.com/library/default.asp?url=/library/en-us/msagent/peedylst_53xw.asp.]
Figure 27.26 Code that defines Microsoft Agent’s actions when Peedy first appears on the screen.
Lines 188–193 call the Speak method twice to specify what Peedy says.The first argument passed to the Speak method contains the string thatPeedy speaks by using the computer’s speakers. This string also appears ina conversation bubble above his head. You have already seen an example ofthis in Fig. 27.4. In this example, you provide the instructions for using thePhone Book application. [Note: The second argument to the Speak methodis a string specifying the location of an audio file to be played. You pass theempty string so that no file will be played.] After Peedy displays theinstructions, he is positioned in his rest pose (Fig. 27.5) by passing string"RestPose" (line 195) to the Play method.
8. Enabling and disabling controls. Now that you have made the Agent char-acter available by clicking the Call Peedy Button, you should disable theButton and allow the ComboBox to be used. Add lines 197–198 of Fig. 27.27to your Click event handler. These lines set the Call Peedy Button’sEnabled property to false and cboName’s Enabled property to true whilethe Agent character is shown. Recall that the ComboBox is disabled bydefault. You disable the Call Peedy Button because Peedy is on the screenat this point.
9. Saving the project. Select File > Save All to save your modified code.
(cont.)
Make Peedy wave
Make Peedy speak instructions
Make Peedy rest
GUI Design Tip
Use Microsoft Agent character ges-tures to indicate actions the usershould take, or a response to anaction the user has already taken.
738 Introducing Multimedia Using Microsoft Agent Tutorial 27
Now that you have written code to display Peedy on the screen and have himspeak to the user, you must insert code to allow Peedy to respond to user input.Peedy needs to display and read a phone number when the user selects a validname, either by speaking, by using the context menu or by using the ComboBox. Youwill enable these features in the following box.
Completing the PhoneBook Application
Figure 27.27 Disable btnCall after Peedy has appeared on the screen.
(cont.)
Enable the ComboBox anddisable the Call Peedy Button
1. Defining the objMainAgent_Command event handler. The event handlerobjMainAgent_Command executes when the Command event is raised. Recallthat this happens when a user speaks a command to the Agent character byusing the microphone (while pressing the Scroll Lock key). The event is alsoraised if the user selects a command from Peedy’s context menu. You cangenerate the event handler by clicking the Events Button in the Propertieswindow, then double clicking the Command event. Be sure to add the com-ments and break the header as shown in Fig. 27.28 so that the line numbersin your code match those presented in this tutorial. Then, add lines 207–209of Fig. 27.28 to this event handler.
Lines 208–209 declare the objCommand variable, which stores values ofthe AgentObjects.IAgentCtlUserInput type. This object is used toretrieve the commands that users give Peedy. Notice that you use the castoperator to convert the user input to the AgentObjects.IAgentCtlUse-rInput type. You do this to access the name of the command the Agentreceived.
2. Interpreting user input and displaying a phone number. Add lines 211–228of Fig. 27.29 to your code. Line 211 creates a counter variable that is used ina for statement (lines 216–228). The body of this statement contains an ifstatement (lines 219–227). The code inside the if statement executes if theuser’s command matches one of the voice-enabled commands you addedearlier by using the m_strNameList array.
Figure 27.29 Finding the spoken or selected name.
Using the counter variable, the for statement iterates through each indexof the array, comparing the command name (for example, John) to the namein the array. If the names match, the body of the if statement plays Peedy’sThink animation (line 221) and uses Peedy to speak and display therequested phone number (lines 222–224). After speaking the phone number,Peedy smiles (the Pleased animation played in line 226).
3. Creating the cboName_SelectedIndexChanged event handler. You mustnow write code to display a phone number if the user chooses a name fromthe cboName ComboBox. Double click the cboName ComboBox in the Win-dows Form Designer to generate the cboName_SelectedIndexChangedevent handler, and enter code view. Be sure to add the comments and breakthe header as shown in Fig. 27.30 so that the line numbers in your codematch those presented in this tutorial. Then, add lines 237–254 in Fig. 27.30to the event handler.
This ComboBox allows users who are unable to access the voice-recogni-tion engines to use the application. Much as in the Command event handler,lines 241–254 search the array for a string that matches the selected item inthe ComboBox. If a match is found, then Peedy’s Think animation plays(Fig. 27.7), and the phone number is provided. After Peedy states the num-ber, his Pleased animation plays, indicating that he is content with his abil-ity to provide the user with the correct phone number (Fig. 27.9).
The cboName_SelectedIndexChanged method executes code similar tothe objMainAgent_Command event handler. The only difference is that theCommand method is invoked by verbal requests (or by using the contextmenu), whereas this method is invoked by selecting items in the cboName.
(cont.)
Peedy thinks and speaks thetelephone number, if found
4. Resetting the application’s controls when Peedy is hidden. You might havenoticed that you can choose Hide from Peedy’s context menu (when youright click Peedy). When the user hides the Peedy character, you shouldenable the Call Peedy Button to allow the user to make Peedy reappear.Also, to prevent the user from selecting a name from the ComboBox whilePeedy is hidden, you should disable the ComboBox. When the user selectsHide, a HideEvent is raised. To generate the HideEvent event handler, clickthe Events Button in the Properties window, then double click theHideEvent event. Be sure to add the comments and break the header asshown in Fig. 27.31 so that the line numbers in your code match those pre-sented in this tutorial. Then, add lines 263–264 of Fig. 27.31 to the event han-dler. These lines enable the Call Peedy Button and disable the ComboBox.
Figure 27.31 Microsoft Agent HideEvent event handler.
(cont.)
Peedy thinks and speaks thetelephone number, if found
Figure 27.32 presents the source code for the Phone Book application. Thelines of code that contain new programming concepts that you learned in this tuto-rial are highlighted.
5. Running the application. Select Debug > Start to run your application.Click the Call Peedy Button to display the Peedy Microsoft Agent charac-ter. Use Peedy to determine the phone numbers stored in the application.
6. Closing the application. Close your running application by clicking its closebox.
7. Closing the IDE. Close Visual Studio .NET by clicking its close box.
1 using System;2 using System.Drawing;3 using System.Collections;4 using System.ComponentModel;5 using System.Windows.Forms;6 using System.Data;78 namespace PhoneBook9 {
10 /// <summary>11 /// Summary description for FrmPhoneBook.12 /// </summary>13 public class FrmPhoneBook : System.Windows.Forms.Form14 {15 // Label displays instructions16 private System.Windows.Forms.Label lblInstructions;1718 // Button calls Peedy19 private System.Windows.Forms.Button btnCall;2021 // ComboBox to select name to call22 private System.Windows.Forms.ComboBox cboName;2324 25 2627 /// <summary>28 /// Required designer variable.29 /// </summary>30 private System.ComponentModel.Container components = null;3132 3334 // fill array with people's names35 private string[] m_strNameList =36 { "John", "Jennifer", "Howard" };3738 // fill array with people's phone numbers39 private string[] m_strNumberList =40 { "(555) 555-9876", "(555) 555-1234", "(555) 555-4567" };4142 public FrmPhoneBook()43 {44 //45 // Required for Windows Form Designer support46 //
Figure 27.32 Phone Book application code. (Part 1 of 4.)
(cont.)
// MSAgent to use the Peedy agentprivate AxAgentObjects.AxAgent objMainAgent;
private AgentObjects.IAgentCtlCharacter m_objMSpeaker;Variable to hold
744 Introducing Multimedia Using Microsoft Agent Tutorial 27
SELF-REVIEW 1. The event is called when the user hides Peedy.
2. Use the method to relocate the Microsoft Agent character on the screen.
Answers: 1) d. 2) c.
27.5 Wrap-UpIn this tutorial, you were introduced to Microsoft Agent and learned how it can beused to enhance software applications. You learned what Microsoft Agent is used
161 162 163 164 }165166 } // end method objMainAgent_Command167168 // called when selection is made in ComboBox169 private void cboName_SelectedIndexChanged( 170 object sender, System.EventArgs e )171 {172 int intCounter;173174 // search m_strNameList array for selected ComboBox item and175 // return number176 for ( intCounter = 0; intCounter < m_strNameList.Length;177 intCounter++ )178 {179 if ( Convert.ToString( cboName.SelectedItem ) ==180 m_strNameList[ intCounter ] )181 {182 183 184 185 186187 188 }189 }190191 } // end method cboName_SelectedIndexChanged192193 // called when user hides Peedy194 195 196 {197 btnCall.Enabled = true;198 cboName.Enabled = false;199200 } // end method objMainAgent_HideEvent201202 } // end class FrmPhoneBook203 }
Figure 27.32 Phone Book application code. (Part 4 of 4.)
for and how to download all the necessary components. You then wrote code to usea Microsoft Agent character, Peedy the Parrot.
Using Peedy, you created an application that interacts with the user by listeningto the user’s commands and answers using speech. You used the Show and MoveTomethods to show Peedy and move him on the screen. You also used Peedy’s Speakmethod to speak instructions and phone numbers to the user. You then learnedhow to control Peedy’s gestures, such as Wave and Think, not only to entertain theuser but also to signal visually what the application is doing.
Tutorials 28–31 present you with a Web-based bookstore application case study.You will learn how to build an application that can be accessed by using a Webbrowser. In Tutorial 28, you will be introduced to the concept of a multi-tier appli-cation and you will take your first steps toward building a three-tier application,which you will complete in the subsequent tutorials.
SKILLS SUMMARY Displaying the Microsoft Agent to Users■ Load the desired Microsoft Agent, using the Characters.Load method.
■ Call the Microsoft Agent object’s Show method.
Allowing Verbal Communication with the Agent■ Add voice-enabled commands, using the Microsoft Agent’s Commands.Add method.
■ Define the Command event handler.
Causing the Agent to Speak■ Call the Microsoft Agent’s Speak method.
■ Specify the string that Microsoft Agent should speak.
Causing Agent to Perform Actions■ Call the Microsoft Agent character’s Play method.
■ Specify the animation that Microsoft Agent should display.
KEY TERMS Add method of the Commands property—Adds a command to a Microsoft Agent character.
AgentObjects.IAgentCtlCharacter object—References a Microsoft Agent character.
AgentObjects.IAgentCtlUserInput object—Stores the user input retrieved from aMicrosoft Agent character.
Characters property of the Microsoft Agent control—Used to access a specific MicrosoftAgent character.
Command event of the Microsoft Agent control—Raised when a user speaks a command to aMicrosoft Agent character or selects a command from a character’s context menu.
Commands property of class IAgentCtlCharacter—Sets which commands the MicrosoftAgent character can understand as input from the user.
Convert.ToInt16 method—Converts data to type short.
Cursor.Position property—Property containing the x- and y-coordinates of the mousecursor on the screen (in pixels).
HideEvent event—Event raised when a Microsoft Agent character is hidden.
IAgentCtlCharacter object—References a Microsoft Agent character.
IAgentCtlUserInput object—Stores the user input retrieved from a Microsoft Agentcharacter.
interactive animated characters—The Microsoft Agent technology adds such characters toWindows applications and Web pages. These characters can interact with the userthrough mouse clicks and microphone input.
Microsoft Agent—Adds interactive animated characters to Windows applications and Webpages.
MoveTo method—Relocates the Microsoft Agent character on the screen.
multimedia—The use of various media, such as sound, video and animation, to create contentin an application.
746 Introducing Multimedia Using Microsoft Agent Tutorial 27
Play method—Plays a Microsoft Agent character animation.
short type—Holds small integer values.
Show method—Displays a Microsoft Agent character on the screen.
Speak method—Has the Microsoft Agent character speak text to the user.
speech-recognition engine—Translates vocal sound input from a microphone into a languagethat the computer understands.
status box—A box that appears below a Microsoft agent character that displays informationabout the character’s actions.
text-to-speech engine—Application that translates typed words into spoken sound that usershear through headphones or speakers connected to a computer.
GUI DESIGNGUIDELINES
Microsoft Agent Control■ Locate the Microsoft Agent character near the application’s Form.
■ Use Microsoft Agent character gestures to indicate actions the user should take, or aresponse to an action the user has already taken.
CONTROLS, EVENTS,PROPERTIES &
METHODS
Convert The Convert class converts the value of a type to another type.
■ Method
ToInt16—Converts the value from another type to type short.
IAgentCtlCharacter This class is used to represent the Agent character that is used in the application.
■ Property
Commands—Contains the commands the character will recognize.
■ Methods
Show—Displays the character on the screen.MoveTo—Moves the character to a specified location on the screen.Play—Plays character animations.Speak—Specifies the text to be spoken by the character.Commands.Add—Adds a new command to the command list for the Agent object.
IAgentCtlUserInput This class is used to retrieve commands from users.
■ Property
Name—Retrieves the name of the command given by the user.
Microsoft Agent Control This control is used to create and manipulate the multimedia features of a Microsoft Agent character.
■ In action
■ Events
Command—Raised when a user gives the Microsoft Agent character a verbal command orselects an option from the character’s context menu.
HideEvent—Raised when a user hides the Microsoft Agent character.
■ Property
Location—Specifies the location of the Microsoft Agent control on the Form.
Characters.Load—Loads a character into the Microsoft Agent control.
MULTIPLE-CHOICEQUESTIONS
27.1 The method is used to specify what the Microsoft Agent will say.
27.2 The method is used to activate a Microsoft Agent character’s animation.
27.3 The MoveTo method takes three arguments. What do the first two arguments represent?
27.4 Which method of IAgentCtlCharacter displays the Microsoft Agent character on thescreen?
27.5 Use the event handler to execute code when users click Hide in the Agentcharacter context menu.
27.6 The Add method of the Commands property .
27.7 The event handler controls what occurs when users speak to the Agent.
27.8 specifies the x-coordinate of the mouse cursor on the screen.
27.9 Specifying as a parameter to Peedy’s Play method causes him to smile.
27.10 Specifying as a parameter to Peedy’s Play method causes him to rest.
EXERCISES 27.11 (Appointment Book Application Using Microsoft Agent) Write an application thatallows users to add appointments to an appointment book that uses Microsoft Agent(Fig. 27.33). When users speak a person’s name, Merlin returns the time and date of theappointment that users have with that person. If users say “Today,” Merlin returns a list ofthe users’ appointments for the day.
a) Copying the template to your working directory. Copy the directory C:\Examples\Tutorial27\Exercises\AppointmentBook to your C:\SimplyCSP directory.
a) Speak b) Say c) Command d) Voice
a) Show b) Play c) Speak d) Appear
a) The direction in which the Agent should move (left, right, up, down).b) The name of the character and its position.c) The x-coordinate and y-coordinate of the location to which the Agent should move.d) The name of the character and the direction of movement.
a) Play b) Show c) Speak d) Appear
a) Hide b) HideEvent c) Command d) Disappear
a) adds a new command to the command listb) joins two commands togetherc) displays the Commands pop-up windowd) Both a and c
a) Command b) ClickEvent
c) Click d) SelectedIndexChanged
a) Cursor.Location.X b) Cursor.Position.X c) Mouse.Location.X d) Mouse.Position.X
748 Introducing Multimedia Using Microsoft Agent Tutorial 27
Figure 27.33 Appointment Book GUI.
b) Opening the application’s template file. Double click AppointmentBook.sln in theAppointmentBook directory to open the application.
c) Downloading the Merlin Microsoft Agent. Download the Merlin.acs character filefrom the Microsoft Web site.
d) Adding the Agent Control to the Form. Add the Microsoft Agent control to the Form.Rearrange and comment the control declaration appropriately.
e) Creating instance variables. Create three instance variables of the ArrayList typeto store the date, time and person with which the user has an appointment. Create aninstance variable of the AgentObjects.IAgentCtlCharacter type (as you did in thePhone Book application).
f) Defining the FrmAppointments_Load event handler. Load Merlin’s character file,display him on the screen and add the Today command to the command list.
g) Defining the btnAdd_Click event handler. Define this event handler so that theinformation provided by the user is added to its corresponding ArrayList. TheAppointment With: TextBox input should be added to the ArrayList containing thenames of people with whom the user has an appointment. The input for the appoint-ment date and time should also be added to their respective ArrayLists. Display anerror message if the user leaves the Appointment With: or the Appointment Time:TextBox empty.
h) Adding voice-enabled commands. Within the btnAdd_Click event handler, add avoice-enabled command to allow a user to speak the name of the person with whomthe user has an appointment to the command list. This allows a user to check whetherthere is an appointment with someone by speaking the person’s name. The commandshould also appear in the Commands context menu.
i) Defining the Agent’s Command event handler. As you did in the Phone Book applica-tion, define what occurs when a user speaks or selects a command. If the user speci-fies the Today command, Merlin should tell the user the names of all the people withwhom the user has an appointment today. If the user specifies a specific name, Merlinshould state the time and date at which the user has an appointment with this person.If the user did not schedule any appointments, Merlin should inform the user that noappointments were scheduled.
j) Running the application. Select Debug > Start to run your application. Add severalappointments to ensure that your application runs correctly as in Fig. 27.33.
k) Closing the application. Close your running application by clicking its close box.
l) Closing the IDE. Close Visual Studio .NET by clicking its close box.
27.12 (Craps Game Application Enhancement Using Microsoft Agent) Modify the CrapsGame application from Tutorial 16 to include a Microsoft Agent character (Fig. 27.34).
a) Copying the template to your working directory. Copy the directory C:\Examples\Tutorial27\Exercises\CrapsGameEnhancement to your C:\SimplyCSP directory.
b) Opening the application’s template file. Double click CrapsGame.sln in the Craps-GameEnhancement directory to open the application.
c) Downloading the Genie Microsoft Agent. Download the Genie.acs character filefrom the Microsoft Web site.
d) Adding the Agent control to the Form. Add the Microsoft Agent control to the Form.Rearrange and comment the control declaration appropriately.
e) Creating an instance variable. Create an instance variable of the AgentOb-jects.IAgentCtlCharacter type (as you did in the Phone Book application).
f) Defining the FrmCrapsGame_Load event handler. Load Genie’s character file, anddisplay him on the screen.
g) Modifying the btnPlay_Click event handler. Add code to the btnPlay_Clickevent handler to control the Agent. When the user wins the game, Genie should playhis Pleased animation and congratulate the user. If the user loses, Genie should playhis Confused animation and say that the user lost. If the user neither wins nor loses,Genie should tell the user to roll again. Make sure to reset him to his RestPose afterhe plays any animation.
h) Defining the btnRoll_Click event handler. Add code to the btnRoll_Click eventhandler to control the Agent. If users “make their point,” Genie should play hisPleased animation and state that the user won. If the user rolls a 7, Genie shouldplay his Confused animation and say that the user lost. Otherwise, Genie should tellthe user to roll again.
i) Defining the btnInstructions_Click event handler. Define the event handlerbtnInstructions_Click to make Genie introduce himself to the user. Genie shouldthen explain the rules to the game of craps.
j) Running the application. Select Debug > Start to run your application. Play severalgames to ensure that your application runs correctly. Also be sure to test the Instruc-tions Button.
k) Closing the application. Close your running application by clicking its close box.
l) Closing the IDE. Close Visual Studio .NET by clicking its close box.
27.13 (Security Panel Application Enhancement Using Microsoft Agent) Modify theSecurity Panel application from Tutorial 12 to include Microsoft Agent (Fig. 27.35).
Figure 27.35 Enhanced Security Panel application.
a) Copying the template to your working directory. Copy the directory C:\Examples\Tutorial27\Exercises\SecurityPanelEnhancement to your C:\SimplyCSPdirectory.
750 Introducing Multimedia Using Microsoft Agent Tutorial 27
b) Opening the application’s template file. Double click SecurityPanel.sln in theSecurityPanelEnhancement directory to open the application.
c) Downloading the Robby Microsoft Agent. Download the Robby.acs character filefrom the Microsoft Web site.
d) Adding the Agent control to the Form. Add the Microsoft Agent control to the Form.
e) Creating an instance variable. Create an instance variable of the AgentOb-jects.IAgentCtlCharacter type (as you did in the Phone Book application).Rearrange and comment the control declarations appropriately.
f) Defining the FrmSecurityPanel_Load event handler. Load Robby’s character file,and display him on the screen. Command Robby to tell users to input their accesscodes.
g) Modifying the btnEnter_Click event handler. Add code to the btnEnter_Clickevent handler to use the Microsoft Agent. If the user enters a valid access code,Robby should welcome the user and state the type of employee that the access coderepresents. If the access code is invalid, then Robby should state that an invalid codewas provided and that access is denied.
h) Running the application. Select Debug > Start to run your application. Enter vari-ous correct and incorrect codes to ensure that your application runs correctly.
i) Closing the application. Close your running application by clicking its close box.
j) Closing the IDE. Close Visual Studio .NET by clicking its close box.
What does this code do? 27.14 After the user clicks the Call Button, what does the following event handler do?
What’s wrong with this code? 27.15 Find the error(s) in the following code. The event handler should have an agent objectappear and say, “Hello, my name is Merlin.” This should happen when the user clicks the CallButton.
Programming Challenge 27.16 (Car Payment Calculator Application Enhancement Using Microsoft Agent)Enhance the Car Payment Calculator application from Tutorial 9 to use the MicrosoftAgent Robby. When the application is run, Robby should appear on the screen and wave tousers. He should then explain what the application does. After users have entered informa-tion into each field of the Car Payment Calculator and clicked the Calculate Button, Robbyshould speak the calculated payment amounts and the number of months for which theywere calculated. Copy the directory C:\Examples\Tutorial27\Exercises\CarPayment-CalculatorEnhancement to your C:\SimplyCSP directory. Double click the CarPayment-Calculator.sln file to open the application in Visual Studio .NET.
