Department of Computer Science and Engineering The University of Texas at Arlington
Easy Office Solutions System Test Plan
Team Members Brent Burns Shawn Gilleland Gideon Miller Miguel Sanchez Sponsor Dr. Darin Brezeale
Last Updated: 7 April 2013 @ 11:31:00 PM
System Test Plan Project Name: Supply Fetch
Easy Office Solutions i Table of Contents
Table of Contents 1 Introduction .................................................................................................................... 1
1.1 Document Overview ....................................................................................................... 1 1.2 Product Overview ........................................................................................................... 1 1.3 Project Scope ................................................................................................................. 2 1.4 Acronym Definitions ....................................................................................................... 3
2 References ...................................................................................................................... 4
2.1 Overview ........................................................................................................................ 4 2.2 System Requirements Specification ................................................................................ 4 2.3 Architecture Design Specification ................................................................................... 8 2.4 Detailed Design Specification........................................................................................ 12
3 Test Items ..................................................................................................................... 16
3.1 Overview ...................................................................................................................... 16 3.2 Relational Diagram ....................................................................................................... 17 3.3 Hardware Tests ............................................................................................................ 18 3.4 Unit Tests ..................................................................................................................... 18 3.5 Component Tests ......................................................................................................... 22 3.6 Integration Tests .......................................................................................................... 24 3.7 System Validation Tests ................................................................................................ 26
4 Risks .............................................................................................................................. 27
4.1 Overview ...................................................................................................................... 27 4.2 Risk Table ..................................................................................................................... 27
5 Features To Be Tested ................................................................................................... 28
5.1 Overview ...................................................................................................................... 28 5.2 Customer Requirements ............................................................................................... 28 5.3 Performance Requirements .......................................................................................... 29 5.4 Maintenance and Support Requirements ..................................................................... 30 5.5 Other Requirements ..................................................................................................... 31
6 Features Not To Be Tested ............................................................................................ 32
6.1 Overview ...................................................................................................................... 32 6.2 Customer Requirements ............................................................................................... 32 6.3 Packaging Requirements .............................................................................................. 32 6.4 Safety Requirements .................................................................................................... 32 6.5 Maintenance and Support Requirements ..................................................................... 33 6.6 Other Requirements ..................................................................................................... 33
7 Overall Test Strategy ..................................................................................................... 34
7.1 Overview ...................................................................................................................... 34 7.2 Configurations .............................................................................................................. 34 7.3 Strategy ........................................................................................................................ 34
System Test Plan Project Name: Supply Fetch
Easy Office Solutions ii Table of Contents
7.4 Metrics ......................................................................................................................... 35 7.5 Regression .................................................................................................................... 35
8 Acceptance Criteria ....................................................................................................... 36
8.1 Overview ...................................................................................................................... 36 8.2 Hardware Testing ......................................................................................................... 36 8.3 Unit (Module) Testing ................................................................................................... 36 8.4 Component (Subsystem) Testing .................................................................................. 37 8.5 Integration (Layer) Testing ............................................................................................ 37 8.6 System Verification Testing .......................................................................................... 37
9 Test Deliverables ........................................................................................................... 38
9.1 Overview ..................................................................................................................... 38 9.2 Deliverables ................................................................................................................. 38
10 Test Schedule ................................................................................................................ 40
10.1 Overview ...................................................................................................................... 40 10.2 Test Schedule ............................................................................................................... 40
11 Approvals ...................................................................................................................... 41
System Test Plan Project Name: Supply Fetch
Easy Office Solutions iii Table of Contents
Document Revision History
Revision Number Revision Date Description Rationale
0.1 3/27/2013 All sections merged for First Draft First Draft Submission
1.0 3/28/2013 Final revisions for First Draft First Draft Submission
2.0 4/7/2013 Final revisions Baseline Submission
System Test Plan Project Name: Supply Fetch
Easy Office Solutions iv Table of Contents
List of Tables Table 1-1 Acronym Definitions ....................................................................................................3
Table 2-1 Customer Requirements ..............................................................................................4
Table 2-2 Packaging Requirements ..............................................................................................5
Table 2-3 Performance Requirements .........................................................................................5
Table 2-4 Safety Requirements....................................................................................................6
Table 2-5 Maintenance and Support Requirements ....................................................................6
Table 2-6 Other Requirements ....................................................................................................7
Table 2-7 Inter-Subsystem Data Flow Descriptions ....................................................................10
Table 2-8 Requirements Mapping by Layer ...............................................................................11
Table 2-9 Inter-Module Data Flow Descriptions.........................................................................14
Table 2-10 Requirements Traceability Matrix by Module ..........................................................15
Table 3-1 Hardware Tests ..........................................................................................................18
Table 3-2 Multimodal User Interface Layer Unit Tests ...............................................................20
Table 3-3 Central Processing Layer Unit Tests ...........................................................................20
Table 3-4 Storage Layer Unit Tests ............................................................................................21
Table 3-5 Indication Layer Unit Tests .........................................................................................21
Table 3-6 Multimodal User Interface Layer Subsystem Tests .....................................................23
Table 3-7 Central Processing Layer Subsystem Tests .................................................................23
Table 3-8 Storage Layer Subsystem Tests ..................................................................................24
Table 3-9 Indication Layer Subsystem Tests...............................................................................24
Table 3-10 Multimodal User Interface Layer Tests .....................................................................24
Table 3-11 Central Processing Layer Tests .................................................................................25
Table 3-12 Storage Layer Tests ..................................................................................................25
Table 3-13 Indication Layer Tests ..............................................................................................25
Table 3-14 System Validation Tests ...........................................................................................26
Table 4-1 Risk Table ..................................................................................................................27
Table 10-1 Test Schedule...........................................................................................................40
System Test Plan Project Name: Supply Fetch
Easy Office Solutions v Table of Contents
List of Figures Figure 1-1 Product Concept .........................................................................................................2
Figure 2-1 Architectural Layers ....................................................................................................8
Figure 2-2 Architectural Layers and Subsystems ..........................................................................9
Figure 2-3 Architectural Layers and Subsystems with Modules .................................................12
Figure 3-1 Relational Diagram ...................................................................................................17
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 1 Introduction
1 Introduction
1.1 Document Overview
The purpose of this System Test Plan (STP) document is to provide detailed testing procedures to ensure that the Supply Fetch system meets all necessary requirements and is a usable and working end-product when delivered to our customers, Dr. Brezeale and the CSE Department.
In the Architecture Design Specification (ADS) document, the entire Supply Fetch system was broken down into four general layers each containing a various amount of subsystems. In the Detailed Design Specification (DDS) document, these subsystems were further divided into smaller and separate modules each having specific tasks and purposes. The DDS document also began laying the ground work for this test plan in the Quality Assurance section.
This STP document will detail all testing procedures for every layer, subsystem, module, and all interactions in between. This plan will also provide the metrics to ensure that the Supply Fetch system meets all critical requirements defined in the SRS document. The various testing procedures outlined in this document include hardware, module (unit), subsystem (component), layer (integration), and system verification testing.
1.2 Product Overview
The Supply Fetch product is a voice controlled system used in an office supply room that allows a user to verbally state the item they seek and the corresponding drawer with that item is indicated through a lighting system.
The purpose of Supply Fetch is to provide a more efficient method of searching for office supply items versus the traditional method of manually looking at labels and searching through various drawers. This product basically fulfills the problem of “show me the item I am looking for” in a timely and efficient manner. The Supply Fetch system would appeal to any individual, group, organization, or company that routinely uses an office supply room to acquire supplies.
The Supply Fetch system will indicate the location of the drawer by means of a lighting system underneath the columns of drawers. When a user requests an item, the light below the correct column will glow in one of four different colors. The emitted color corresponds to small colored labels attached on every drawer to correctly identify the drawer containing the sought item. Figure 1-1 on the following page provides a visualization of the Supply Fetch system.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 2 Introduction
The Supply Fetch system has four main components: (1) the microphone, (2) the interface component, (3) the indication controller, and (4) the light indication component. These components are not to be confused with the Architectural Layers or subsystems. Supply Fetch does not monitor or handle any type of inventory (e.g. system will not notify user when the supply of pencils has been exhausted).
Figure 1-1 Product Concept
1.3 Project Scope
As previously stated, Supply Fetch is a system that will locate an item in a supply room when a user specifies the name of the item by voice. The project concept originated from Dr. Darin Brezeale at the University of Texas at Arlington who conceived the idea to aid in the CSE Department’s supply room. Therefore we plan to integrate our finished product into the CSE Department’s supply room located on the 6th floor of the Engineering Research Building at UTA. So our immediate audience that will utilize the Supply Fetch system is any UTA staff and faculty member who uses this supply room.
Voice Acquisition
Component
Interface Component
Indication Controller Component
Light Indication Component
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 3 Introduction
1.4 Acronym Definitions
Below are some frequently used acronyms that are used throughout this document.
Term Definitions
ADS Architecture Design Specification
CP Layer Central Processing Layer
CSE Computer Science and Engineering
DB Database
DBM Database Manager
DDS Detailed Design Specification
EOS Easy Office Solutions
ERB Engineering Research Building
GUI Graphical User Interface
HCG Hardware Communication Gateway
I2C I2C Bus – Two Wire Interface
IC Indication Controller
IR Infrared
ISR Item Services and Reporting
LED Light Emitting Diode
LIC Light Indication Component
MUI Layer Multimodal User Interface Layer
OS Operating System
RGB Red Green Blue (multi-color) LED
SRS System Requirements Specification
STP System Test Plan
STT Speech-to-Text
UI User Interface
Table 1-1 Acronym Definitions
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 4 References
2 References
2.1 Overview
This STP Document utilizes elements from the SRS, ADS, and DDS documents. In order to ensure that the Supply Fetch system is tested and verified accurately and to the fullest extent possible, content from all of these documents must be taken into consideration when developing this test plan. The following subsections contain the overviews of critical sections of the aforementioned documents and were deemed of high importance for the Supply Fetch system.
2.2 System Requirements Specification
The SRS document defined all requirements deemed necessary by Easy Office Solutions, Dr. Brezeale, Mr. O’Dell, or peer input. An overview of all 28 requirements will be given below with a brief description and an assigned priority. For further detail reference the SRS document.
2.2.1 Customer Requirements
This subsection defines the requirements the customer has specified as being highly important. The customer is the EOS team sponsor Dr. Darin Brezeale. The requirements below cannot be changed without approval from the customer.
SRS
No. Requirement Description Priority
3.1 Size of Supply Fetch The Supply Fetch system should be non-invasive and should not interfere
with regular office room activities as a result of its size. 1 – Critical
3.2 Simple User Interface The user interface should be simple and easy to use. It should also provide
fast and seamless functionality and activate the microphone for voice to text processing with at most a single button.
1 – Critical
3.3 Prepping of the System The average user should not need to “prep” or “train” the Supply Fetch
system beforehand, like dictation software. 1 – Critical
3.4 Speech Processing
System The Supply Fetch system will use a microphone to capture the user’s voice
for voice to text processing. 1 – Critical
3.5 System Should Find Item
Requested by User The Supply Fetch system should indicate to the user the correct location of
the item they requested. 1 – Critical
Table 2-1 Customer Requirements
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 5 References
2.2.2 Packaging Requirements
This subsection defines the packaging requirements for the Supply Fetch system that describe system contents, setup, and operating requirements. The Supply Fetch system will be designed for easy setup, maintenance, and use. The system will only require attaching four devices together and the software will come preinstalled.
SRS
No. Requirement Description Priority
4.1 Product Will Include One
Interface Device
The Supply Fetch system will include a device with a screen and controls that will allow a user to activate the voice processing and mapping items.
Interface software will come preinstalled. 2 – High
4.2 Product Will Include One
Microphone The Supply Fetch system will include a microphone that will capture the
user’s voice. This component will attach to the interface device. 2 – High
4.3 Product Will Include One
Indicator Controller Device
The Supply Fetch system will include a device that communicates with the tags to indicate the correct drawer. The controller device will be attached
to the interface component. 3 – Moderate
4.4 Product Will Include the
Necessary Indicator Devices
The Supply Fetch system will include the necessary indicator devices that act as tags to indicate where items are. These devices will each contain a
LED light. 3 – Moderate
Table 2-2 Packaging Requirements
2.2.3 Performance Requirements
This subsection defines the performance requirements that the Supply Fetch system should meet. These requirements are based on the aspects of both speech processing and item finding procedures so that issues are handled appropriately.
SRS
No. Requirement Description Priority
5.1 Accented Speech The Supply Fetch system should process accordingly for various user
accents. 3 – Moderate
5.2 Singular vs. Plural Item
Request
The Supply Fetch system should accommodate for referencing an office supply in singular, plural, and other common names (i.e. “pen” or “pens”
or “ballpoints”). 3 – Moderate
5.3 Background Noise
Reduction The Supply Fetch system should decipher between the user’s voice and
background noise. 3 – Moderate
5.4 Voice Detection Wait
Time The Supply Fetch system should remain in an active listening state for a
limited amount of time after the item search is activated. 2 – High
5.5 Location Revealing
Response Time The Supply Fetch system should reveal the location of the item within a
reasonable amount of time. 2 – High
5.6 Indication Method Hold
Time The Supply Fetch system indicators should flash for a reasonable amount
of time once the item has been found 2 – High
5.7 Item Request Failure The Supply Fetch system should provide an alternative input method (e.g.
GUI keyboard input, dropdown choices etc.) if the system continuously fails at identifying an item.
2 – High
Table 2-3 Performance Requirements
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 6 References
2.2.4 Safety Requirements
This subsection defines the safety requirements from the perspective of the user, software, hardware, and environment of the embedded system. The system will need to comply with safety procedures when interaction occurs between administrative access, drawer opening/closing, hardware edges, wiring, and on/off switches of the product.
SRS
No. Requirement Description Priority
6.1 Non-Hazardous
Equipment
The Supply Fetch system should not interfere with the regular space or drawer use in such a way that it becomes hazardous to users (i.e. should not expose wire that is not properly insulated or hardware objects with
rough edges or texture that can cause damage to either the user or drawer).
1 – Critical
6.2 Administrator Interface The Supply Fetch system should only allow an administrator to make
customizations, change settings and add or remove tags/items from the system.
2 – High
Table 2-4 Safety Requirements
2.2.5 Maintenance and Support Requirements
This subsection defines the maintenance and support that will be provided for the delivered product. Documentation troubleshooting, startup, and source code will be provided for quick customizations and referencing; however, any major support will most likely have to come from the supplier.
SRS
No. Requirement Description Priority
7.1 Start-Up Guide The Supply Fetch system should include a guide that will provide the user
with basic instructions on product start-up along with tips for troubleshooting.
2 – High
7.2 Testing
The Supply Fetch system should be tested for performance and correct output delivery in its practice and final environment. Testing, for example,
may consist of having the CSE department staff members participate in item request scenarios.
1 – Critical
7.3 Maintenance and
Support
Team Easy Office Solutions will not be committed to providing maintenance after May 2013 unless members voluntarily decide to
enhance/fix the system functionality when needed. 1 – Critical
7.4 Source Code and Documentation
The Supply Fetch system should include all source code used throughout the project and necessary deliverables for future reference.
1 – Critical
7.5 Testing Feature for the
Indicators
The Supply Fetch system should include a feature that allows the user to test if the indicators are functioning properly. (e.g. Activating this feature would power every LED on so the user could discover if any LEDs are not
functional.)
2 – High
Table 2-5 Maintenance and Support Requirements
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 7 References
2.2.6 Other Requirements
This subsection defines all other relevant requirements that were neither stated nor categorizable in the previous sections.
SRS
No. Requirement Description Priority
8.1 Environment Deployment
The system should be able to be adapted and deployed to other shelves with drawers that are often found in offices and stores.
1 – Critical
8.2 Item Frequency Tracker The system should provide Business Intelligence information to users or
administrators; track the frequency of items requested. 5 – Future
8.3 On/Off Indication The Supply Fetch system should have on/off states that appear obvious to
the user during time of interaction. 2 – High
8.4 On/Off Voice Detection The Supply Fetch system should indicate that it is waiting to receive voice
input from the user for processing the item request. 2 – High
8.5 Power Supply The Supply Fetch system should accommodate for powering components
that need to be attached. 1 – Critical
Table 2-6 Other Requirements
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 8 References
2.3 Architecture Design Specification
The ADS document broke down the Supply Fetch architecture into multiple layers each comprised of various separate subsystems. An overview of the four layers and the 14 subsystems will be given below. For further detail please reference the ADS document.
2.3.1 Layer Overview
Multimodal User Interface Layer
Central Processing Layer
Storage Layer Indication Layer
Figure 2-1 Architectural Layers
Multimodal User Interface Layer The Multimodal User Interface (MUI) Layer will operate on a Windows machine that handles the various forms of input and the output of the system. Both keyboard and voice commands will be formatted accordingly and be sent to the Central Processing (CP) Layer. This layer also expects data from the CP Layer so that process results are displayed to the user via the Application GUI.
Central Processing Layer The Central Processing (CP) Layer handles the process of finding items and maintaining items and tags in the system. Aside from receiving input data from the MUI Layer, the CP Layer will interact with the MUI by providing it with formatted data from process results. Unit identification tags (column number and color) will also be sent to the Indication Layer via the CP Layer while for item lookup processes, the CP Layer will depend on the Storage Layer to get the item unit id tags.
Storage Layer The Storage Layer stores item and tag information which is provided to the CP Layer for performing necessary look up operations. The tag information for each database item will consist of a column number and color. It also receives any lookup commands or queries from the CP Layer.
Indication Layer The Indication Layer is the hardware that will consist of the controller and an attached LED strip array that will indicate to the user the location(s) of a requested item. This layer will not provide any services to any other layer but it will depend on the CP Layer to send the necessary tag(s) that will signal one or more indicators to play a light script.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 9 References
2.3.2 Subsystem Overview
User
Multimodal User Interface Layer
Voice Acquisition
Component
UI Process Manager
GUI
Central Processing Layer
Speech-to-Text
Engine
Item Verification
Storage Layer Indication Layer
Database Manager
Indication Controller Component
System Database
Management
Controller
Item Look-Up
Controller
Item Services and
Reporting
Hardware Communication
Gateway
Hardware
Functionality
C10
C8
I1
Light Indication Component
C7
I2
1……
2.…..
M1
M2
C3
M5
C6
M7
S3
M6
C9
C11
M3M4
C2C1
C5 C4
S2
S1
S4
Output
Figure 2-2 Architectural Layers and Subsystems
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 10 References
Layer Data Element Description
Mu
ltim
od
al U
ser
Inte
rfac
e
Laye
r
M1 Audio collected by the Voice Acquisition Component from the user.
M2 A String which is the result of the Speech-to-Text system matching Audio taken as input to an item in the database.
M3 String Array type containing the names of all available items in the Supply room which will be loaded to the grammar of the engine
M4 An Event that shows the current state of the GUI.
M5 A Component that updates the state of the GUI.
M6 String type that has the value of the item that is being looked for by the user.
M7 String type that contains the value of an item to be added or removed from the available items for the system.
Cen
tral
Pro
cess
ing
Laye
r
C1 Boolean value that is true if an item was found and false otherwise, used for feedback to the user.
C2 Item array holding the values of all available items in the supply room. It can also provide a Boolean value notifying if an item was correctly added or removed from the system.
C3 Item type containing the value of the item that is being looked for by the user.
C4 Boolean value notifying true if item was found in the database and false if otherwise.
C5 Item type that is to be added or removed from the database.
C6 Boolean value notifying that item was correctly added or removed from the database. Also an Item Array type with the values of all items available in the database.
C7 Void type, just notifies the Hardware communication Gateway to light up all lights in the system for maintenance.
C8 Item type containing value of item being looked for by user.
C9 Item type containing value of item to be added or removed from the database.
C10 Item Array type contains ID value that connects Item to certain drawers, used to like up the drawers holding item that is being looked for.
C11 Item Array type containing ID value of all available item and drawers pairs.
Sto
rage
Laye
r
S1 Item value with correct ID added by database for item. ID maps drawer to item location.
S2 Item Array type holding all available items in the database.
S3 Database query, can add and remove an item from the database.
S4 Result of Database query.
Ind
ica
tio
n
Laye
r I1 String value containing value of script to be run and the ID of the LED unit to light up.
I2 Raw data being sent to the indication unit to be lit up.
Table 2-7 Inter-Subsystem Data Flow Descriptions
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 11 References
2.3.3 Requirements Mapping by Layer
SRS No. Requirement Multimodal
User Interface Layer
Central Processing
Layer
Storage Layer
Indication Layer
3.2 Simple UI X
3.4 Speech Processing X
3.5 Find Item X X X X
5.1 Accented Speech X
5.2 Singular/Plural Item Request X
5.3 Background Noise Reduction X
5.4 Voice Detection Wait Time X
5.5 Location Revealing Response Time X X X X
5.6 Indication Method Hold Time X
5.7 Item Request Failure X
6.2 Admin Interface X X X
7.5 Indicator Testing Feature X X X
8.3 On/Off Indication X
8.4 On/Off Voice Detection X
Table 2-8 Requirements Mapping by Layer
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 12 References
2.4 Detailed Design Specification
The DDS document took the subsystems from the ADS document and further divided them into smaller and separate modules each having specific tasks and purposes. An overview of the 33 modules will be given below. For further detail please reference the DDS document.
2.4.1 Module Overview
User
Multimodal User Interface Layer
Voice Acquisition
Component
UI Process Manager
GUI
Central Processing Layer
Speech-to-Text Engine
Item Verification
Storage Layer Indication Layer
Database Manager
System Database
Management Controller Item Look-Up Controller
Item Services and Reporting
Hardware Communication Gateway
Hardware Functionality
C20
Light Indication Component
1……
2.…..
C13
S5
C19
C5
C12
Input
Command Handler Module
I2C BlinkM Module
Serial Communication(USB Port)
Indication Controller Component
Command Interpreter
Module
I2C-Command Sender Module
I2C Communication(I2C Bus)
Location Translation
ModuleI1
I2
I3
I4
Test LED Module
Item Locator Module
ADO.NET Connection
Item Finder Module
Item Finder Module
Item Adder
Module
C15
Item Adder
Module
Item Remover Module
Item Loader Module
Indicator Tester
Module
Item Adder Module
Item Remover Module
Item Loader Module
Item Remover Module
Item Loader Module
Item Finder
Module
S3
S4
S8 S9 S12
Database Module
S13 S14
S6
S7 S10
S11
S1S2
C16C18
C7C6
C8 C9C10
C11
C2
C3
C4
C14
Find Item Module
Speech Recognized
Event Module
Get Audio Input Device Module
Update Grammar Module
Activate Listen Mode Module
Present FeedbackModule
Hardware Functionality Test
Module
Speech Recognized Handler Module
GUI Event Handler Module
M2
M15
C17
Item Inventory Management
Module
M9
M4
M3
M5
M7
M10
C1
M6
M8
M18
M17
M16M14
Get Event Data Module
GUI Event Subscription
Module
M12
M11
M13
M1
Figure 2-3 Architectural Layers and Subsystems with Modules
Note The dataflow numbering and descriptions below are different from the ADS Document, but all general dataflows are consistent between the ADS and DDS dataflow diagrams.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 13 References
Number Description
M1 Stream type and Audio format type, used to by the Speech to Text Engine to set up the audio input device being used.
M2 Object type contains a string that has the value of the word that was recognized by the Speech to Text Engin.
M3 List<List<object>> type containing an Item type at column 0 and Drawer location of that Item type.
M4 String[] containing all the available items in the database, used to build a grammar for the Speech to Text Engine
M5 Method call that notifies the Speech to Text Engine to activate its listening mode.
M6 String type that provides feedback to the user about the add and remove process.
M7 Item type that has the value of the item that the user is looking to find.
M8 String type that provides feedback to the user about the current item that is being looked for by the system.
M9 Item and Drawer[] containing the item and its location(s) to be added or removed from the database.
M10 Item type that has the value of the item being looked for by the user.
M11 Method call sets up the GUI Event Handler Module as a subscriber so that it can receive events that occur in the GUI subsystem.
M12 Generic object type containing information about the action requested to be performed by the user and any relevant data to that action.
M13 Generic object type containing user event information passed to the GUI Event Subscriber so that it can be sent out to outside classes that subscribe to it.
M14 Item type containing value of item to be looked for by the system.
M15 Method call that notifies the system to test all lights available in the system.
M16 Method call that notifies the Item Loader Module to return a Drawer List containing all available items in the system.
M17 Item and Drawer List type that contains value of item to be removed and the location(s) it is to be removed from.
M18 Item and Drawer List type that contains value of item to be added and the location(s) it is to be added to.
M14 Item type containing value of item to be looked for by the system.
M15 Method call that notifies the system to test all lights available in the system.
M16 Method call that notifies the Item Loader Module to return a Drawer List containing all available items in the system.
M17 Item and Drawer List type that contains value of item to be removed and the location(s) it is to be removed from.
M18 Item and Drawer List type that contains value of item to be added and the location(s) it is to be added to.
C1 A Boolean value representing if an item was successfully added to the database.
C2 A Boolean value representing if an item was successfully removed from the database.
C3 A list of Drawer objects, each containing a list of Item objects that they contain.
C4 A list of Drawer objects that contain the item searched for.
C5 An Item object and a Drawer object representing what drawer to add the item to.
C6 A Boolean value representing if an item was successfully added to the database.
C7 An Item object and a Drawer object representing what drawer to remove the item from.
C8 A Boolean value representing if an item was successfully removed from the database.
C9 A function call to retrieve all items in the database.
C10 A list of Drawer objects, each containing a list of Item objects that they contain.
C11 A function call to perform a test indication system procedure.
C12 A Boolean value representing if the specified item was successfully found in the database.
C13 An Item object representing an item to find in the database.
C14 An Item object representing an item to find in the database.
C15 A list of Drawer objects that contain the item searched for.
C16 An Item object and a Drawer object representing what drawer to add the item to.
C17 An Item object and a Drawer object representing what drawer to remove the item from.
C18 A function call to retrieve all items in the database.
C19 A function call to perform a test system procedure.
C20 An object or set of type drawer is sent to a method within the CH Module. The drawer(s) will have the row and column properties so that they can be processed separately.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 14 References
S1 A Boolean value representing if an item was successfully added to the database.
S2 A Boolean value representing if an item was successfully removed from the database.
S3 A list of Drawer objects, each containing a list of Item objects that they contain.
S4 A list of Drawer objects that contain the item searched for.
S5 SQLite query to check if drawer exists. SQLite query to check if item of same name already in drawer. SQLite query to insert item into database.
S6 Result of query to check if drawer exists. Result of query to check if item of same name already in drawer.
S7 Result of query to check if drawer exists. Result of query to check if item of specified name in drawer.
S8 SQLite query to check if drawer exists. SQLite query to check if item of specified name in drawer. SQLite query to remove item from database.
S9 SQLite query to retrieve all drawers. SQLite query to retrieve all items in a drawer.
S10 Result of query to retrieve all drawers. Result of query to retrieve all items in a drawer.
S11 Result of query to retrieve all drawers that contain a specified item.
S12 SQLite query to retrieve all drawers that contain a specified item.
S13 A SQLite query.
S14 A result of a SQLite query.
I1 An integer representing the row needed to be mapped to a specific color code handled by the LED unit .
I2 An color code (hue) as an integer that will fill the placeholder/gap in the command string (“h ”+addr+” ”+hue) in the CH Module.
I3 Send the command string serially to the Arduino once string placeholders have been established.
I4 Send extracted data from string command to I2C sender functions that accept address and hue as parameters in the form of bytes.
I5 Send byte data to the BlinkM module via I2C bus.
Table 2-9 Inter-Module Data Flow Descriptions
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 15 References
2.4.2 Requirements Traceability Matrix by Module
Multimodal User Interface Layer
Central Processing Layer
Storage Layer Indication Layer
Requirement
3.2
Sim
ple
UI
3.4
Spee
ch P
roce
ssin
g
3.5
Fin
d It
em
5.1
Acc
ente
d S
pee
ch
5.2
Sin
gula
r/Pl
ura
l Ite
m R
eque
st
5.3
Bac
kgro
un
d N
oise
Red
ucti
on
5.4
Voi
ce D
etec
tio
n W
ait
Tim
e
5.5
Loca
tion
Res
po
nse
Tim
e
5.6
Ind
icat
ion
Met
ho
d H
old
Tim
e
5.7
Item
Req
ues
t Fa
ilure
6.2
Ad
min
Inte
rfac
e
7.5
Ind
icat
or
Test
ing
Feat
ure
8.3
On
/Off
Ind
icat
ion
8.4
On
/Off
Vo
ice
Det
ecti
on
Sub
syst
em
: M
od
ule
Voice Acquisition Device: Get Audio Input Device X X X X X X
Speech-to-Text Engine: Update Grammar X X X X X X
Speech -to-Text Engine: Activate Listen Mode
Speech -to-Text Engine: Speech Recognized Event X X X X X X X X
UI Process Manager: GUI Event Handler X X X
UI Process Manager: Speech Recognized Handler X X X X
UI Process Manager: Find Item X X
UI Process Manager: Item Inventory Management X
UI Process Manager: Hardware Functionality Test X
GUI: Get Event Data X X X X X
GUI: Present Feedback X X X X X X X
GUI: GUI Event Subscription X X X
Item Look-Up Controller: Item Finder X X
Item Verification: Item Finder X X
Item Verification: Item Locator X X X
Management Controller: Item Adder X
Management Controller: Item Remover X
Management Controller: Item Loader X
Management Controller: Indicator Tester X X
Item Services and Reporting: Item Adder X
Item Services and Reporting: Item Remover X
Item Services and Reporting: Item Loader X
Hardware Functionality: Test LED X X
Database Manager: Item Finder X X
Database Manager: Item Adder X
Database Manager: Item Remover X
Database Manager: Item Loader X
System Database: Database X X X
Hardware Communication Gateway: Command Handler X X X X
Hardware Communication Gateway: Location Translation X X X X
Indication Controller: Command Interpreter X X X
Indication Controller: I2C-Command Sender X X X
Light Indication Component: I2C BlinkM Module X X X
Table 2-10 Requirements Traceability Matrix by Module
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 16 Test Items
3 Test Items
3.1 Overview
This section provides details on the different phases of testing necessary to generate a successful and complete product. The Relational Diagram (Figure 3-1) on the following page represents the System Test Plan in a visual flowchart fashion. The STP is divided into 5 testing phases: hardware, unit, component, integration, and system verification. For clarification, units represent the modules, components represent the subsystems, and integration represents the layers.
Each path of the Relational Diagram can be tested concurrently, but must be traversed in order from left to right. Each phase on the path must be successfully completed before moving on to the next phase. The specific tests are described in the following subsections. They describe the inputs, expected outputs/actions, and risk level.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 17 Test Items
3.2 Relational Diagram
Hardware Unit/Module Component/
Subsystem
Layer Integration System
Activate Listen Mode
Update Grammer
Present Feedback
Speech Recognized
Event
GUI Event Subscription
Get Event Data
Speech Recognized
Handler
GUI Event Handler
Item Inventory
Management
Hardware Functionality
Test
Find Item
Item Adder
Item Remover
Item Loader
Indicator Tester
Item Adder
Test LED
Item Loader
Item Finder
Item Remover
Item Finder
Item Locator
Item Adder
Item Remover
Item Loader
Item Finder
Database
Command Handler
Location Translation
Command Interpreter
Get Audio Input Device
I2C-Command Sender
I2C BlinkMLight Indication
Component
Indication Controller
Component
Hardware Comm
Gateway
System Database
Database Manager
Item Verification
Hardware Functionality
Item Services and
Reporting
Management Controller
UI Process Manager
Item Look-Up
Controller
GUI
Speech-To-Text Engine
Voice Acquisition
Component
Multimodal User
Interface
Central Processing
Storage
Indication
Arduino
Microcontroller
Dell Laptop
Dynex USB Microphone
BlinkM LEDs
Figure 3-1 Relational Diagram
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 18 Test Items
3.3 Hardware Tests
Test ID Individual Hardware
Component Input Expected Output/Action Risk
H1 Dell Laptop
User input via keyboard commands, touchpad
commands, and microphone input
Correct feedback/updates to the user via the screen (GUI)
Correct commands are sent via a USB cable to the Arduino Uno
Appropriate and correct processing of the Supply Fetch program
High
H2 Arduino Uno Receives a string command
from the Hardware Communication Gateway
Sends low level byte commands to and understood by the BlinkM LEDs
High
H3 Dynex Microphone The user’s vocal input of an
item Delivers formatted speech input to the
Voice Acquisition Component subsystem High
H4 BlinkM RGB LEDs Receives low level byte data
for running programmed script Show hue values in a blinking format High
H5 Ribbon Cable
Properly acts as a physical connection to deliver digital values between the Arduino
Uno and the BlinkM LEDs
All BlinkM LEDs receive a good electric connection form the Arduino Uno via the
Ribbon Cable High
Table 3-1 Hardware Tests
3.4 Unit Tests
3.4.1 Multimodal User Interface Layer Unit Tests
Test ID Module Input Expected Output/Action Risk
UM1 GetAudioInputDev
ice Method call
Stream type containing the audio information to be used.
Low
UM2 GetAudioInputDev
ice Method call
SpeechAudioFormatInfo type containing information about the audio
Low
UM3 UpdateGrammar String array containing items in the
system
Returns nothing
Updates the grammar available to the Voice Recognition Engine.
Medium
UM4 ActivateListenMo
de Method call
Returns nothing
Starts the Voice Recognition Engine to start listening for audio input.
Medium
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 19 Test Items
UM5 SpeechRecognized
Event
Generic object and SpeechRecognizedEventArgs type
containing general information about the event that occurred
Returns nothing
Notifies all outside subsystems listening that an even occurred and provides information
about the event.
High
UM6 GUIEventHandler Generic object action containing an
object type passed from the GUI subsystem
Returns nothing
Takes a generic object type, figures out the action the object is related to, and then pulls relevant information provided by the object
and sends it to the correct module within the UIProcessManager subsystem.
High
UM7 SpeechRecognized
Handler
Generic object sender and EventArgs type containing general information
about the event that occurred
Returns nothing
Sends Item type containing item to look for in the database to the FindItem module.
High
UM8 FindItem Item type containing the information
about the item to be looked for by the system
Returns nothing
Starts the lookup process by sending the item to be looked for to the ItemLookUpController module, once process is done it notifies user
of result.
High
UM9 ItemInventoryMa
nagment
Item type containing information about the item and Location of where the item is going to be added into the
system
Returns nothing
Sends item and location to the MaintanenceController Module to add item
and location to the database. Also updates the current list of items available in the system for
SpeechToTextEngine and GUI.
Medium
UM10 ItemInventoryMa
nagment
Item type containing information about the item and Location of where the item is going to be removed from
the system
Returns nothing
Sends item and location to the MaintanenceController Module to add item
and location to the database. Also updates the current list of items available in the system for
SpeechToTextEngine and GUI.
Medium
UM11 HardwareFunction
alityTest Method Call
Returns nothing
Notifies the ManagementController to start the Hardware Functionality Test process.
Low
UM12 GetEventData Generic object and EventArgs type containing information about the
event that occurred
Returns nothing
Depending on action selected by the user this module sends out an object containing
information about the action selected to the GUIEventSubscription module.
High
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 20 Test Items
UM13 PresentFeedback String containing feedback to the
user from the system
Returns nothing
Prints String containing feedback to the user. Low
UM14 GUIEventSubscript
ion
Generic object encapsulating information about the type of action
that was committed by the user
Returns nothing
Sends generic object is receives to the GUIEventHandler module in the
UIProcessManager
High
Table 3-2 Multimodal User Interface Layer Unit Tests
3.4.2 Central Processing Layer Unit Tests
Test ID Module Input Expected Output/Action Risk
UC1 ILUC-Item Finder Item object with valid name Boolean value representing if item found High
UC2 IV-Item Finder Item object with valid name Boolean value representing if item found High
UC3 IV-Item Locator List of Drawer objects Drawer list sent to HCG-Command Handler High
UC4 MC-Item Adder Item object with valid name and
Drawer object Boolean value representing if item added to
database Medium
UC5 MC-Item Remover Item object with valid name and
Drawer object Boolean value representing if item removed
from database Medium
UC6 MC-Item Loader Function call with no parameters List of Drawer objects containing lists of Item
objects High
UC7 MC-Indicator
Tester Function call with no parameters Call HF-Test LED Low
UC8 ISR-Item Adder Item object with valid name and
Drawer object Boolean value representing if item added to
database Medium
UC9 ISR-Item Remover Item object with valid name and
Drawer object Boolean value representing if item removed
from database Medium
UC10 ISR-Item Loader Function call with no parameters List of Drawer objects containing lists of Item
objects High
UC11 HF-Test LED Function call with no parameters Call HCG-Command Handler Low
Table 3-3 Central Processing Layer Unit Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 21 Test Items
3.4.3 Storage Layer Unit Tests
Test ID Module Input Expected Output/Action Risk
US1 DM-Item Adder Item object with valid name and
Drawer object Boolean value representing if item
added to database Medium
US2 DM-Item Remover Item object with valid name and
Drawer object Boolean value representing if item
removed from database Medium
US3 DM-Item Loader Function call with no parameters List of Drawer objects containing lists of
Item objects High
US4 DM-Item Finder Item object with valid name List of Drawer objects High
US5 SD-Database SQLite query Query result High
Table 3-4 Storage Layer Unit Tests
3.4.4 Indication Layer Unit Tests
Test ID Module Input Expected Output/Action Risk
UI1 Command Handler Drawer objects from the CP
Layer
Send Row numbers to LT Module for mapping
Send command string to controller with address and color
High
UI2 Location Translation Row Number from Command
Handler Numeric hue value representing the row
number for the item High
UI3 Command Interpreter String from application via
serial bus Extract address and hue value from string
and store to byte variables High
UI4 I2C-Command Sender Get byte data as parameters that hold address and color
Send low level byte commands understood by BlinkM modules
High
UI5 I2C BlinkM Receive low level byte data for
running programmed script Show hue values in a blinking format High
Table 3-5 Indication Layer Unit Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 22 Test Items
3.5 Component Tests
3.5.1 Multimodal User Interface Layer Subsystem Tests
Test ID Subsystem Input Expected Output/Action Risk
CM1 VoiceAcquisitionComponent Method calls
Sends out Stream type for audio input source and SpeechAudioFormatInfo
type contain the formatting information about the audio input
source.
Low
CM2 SpeechToTextEngine String array containing names
of all available items in the database
No output
Updates grammar available to the SpeechToText engine.
High
CM3 SpeechToTextEngine
Stream type and SpeechAudioFormatInfo type containing information about the audio input device being
used
No output
Sets the audio input source of the SpeechToText engine
High
CM4 SpeechToTextEngine Method call
No output
Notifies the SpeechToTextEngine to start listening for commands from user
Med
CM5 SpeechToTextEngine Audio input Outputs a string containing the value of
a word recognized by the SpeechToTextEngine.
High
CM6 GUI A String and Int list containing
the available items in the system and their locations
No output
Updates the list of items currently available in the
Low
CM7 GUI String type containing
feedback to user Prints feedback message on the screen.
CM8 GUI User input Generic object containing information
about the action the user wants to commit.
High
CM9 UIProcessManager Generic object containing
information about an action a user wants to commit
The output depends on action requested. The Add item and Remove item actions output an Item type and
Drawer type. The Find item action outputs an Item type. The Hardware
Funtionality test action outputs a function call. And, lastly the Activate
Listen Mode action outputs a function call.
High
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 23 Test Items
CM10 UIProcessManager Generic object containing
information about an action a user wants to commit
Function call requesting all items currently available in the system and
their locations. Medium
CM11 UIProcessManager Boolean value returned from a
process Outputs a string providing feedback
about the end state of a process. Low
Table 3-6 Multimodal User Interface Layer Subsystem Tests
3.5.2 Central Processing Layer Subsystem Tests
Test ID Subsystem Input Expected Output/Action Risk
CC1 Item Look-Up Controller Item object with valid name Boolean value representing if item
found High
CC2 Item Verification Item object with valid name Boolean value representing if item
found, Drawer list sent to HCG-Command Handler
High
CC3 Management Controller Item object with valid name
and Drawer object to add into Boolean value representing if item
added to database Medium
CC4 Management Controller Item object with valid name
and Drawer object to remove from
Boolean value representing if item removed from database
Medium
CC5 Management Controller Function call with no
parameters to Item Loader List of Drawer objects containing lists
of Item objects High
CC6 Management Controller Function call with no
parameters to Indicator Tester Call HF-Test LED Low
CC7 Item Services and Reporting Item object with valid name
and Drawer object to add into Boolean value representing if item
added to database Medium
CC8 Item Services and Reporting Item object with valid name
and Drawer object to remove from
Boolean value representing if item removed from database
Medium
CC9 Item Services and Reporting Function call with no
parameters to Item Loader List of Drawer objects containing lists
of Item objects High
CC10 Hardware Functionality Function call with no
parameters to Test LED Call HCG-Command Handler Low
Table 3-7 Central Processing Layer Subsystem Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 24 Test Items
3.5.3 Storage Layer Unit Subsystem Tests
Test ID Subsystem Input Expected Output/Action Risk
CS1 Database Manager Item object with valid name and
Drawer object to add into Boolean value representing if item
added to database Medium
CS2 Database Manager Item object with valid name and Drawer object to remove from
Boolean value representing if item removed from database
Medium
CS3 Database Manager Function call with no parameters to
Item Loader List of Drawer objects containing lists
of Item objects High
CS4 Database Manager Item object with valid name List of Drawer objects High
CS5 System Database SQLite query Query result High
Table 3-8 Storage Layer Subsystem Tests
3.5.4 Indication Layer Subsystem Tests
Test ID Subsystem Input Expected Output/Action Risk
CI1 Hardware Communication
Gateway List of drawers to activate
Generates a string command that the controller can understand
High
CI2 Indication Controller
Component
String Command containing the address and hue
belonging to each item
Extracted byte data for calling correct LEDs and feeding them the correct I2C commands
High
CI3 Light Indication
Component Byte data used to
program/configure LED Affected LED’s, based on address call, will
blink to certain colors High
Table 3-9 Indication Layer Subsystem Tests
3.6 Integration Tests
3.6.1 Multimodal User Interface Layer Tests
Test ID Input Expected Output/Action Risk
IM1 User input
The output depends on action requested. The Add item and Remove item actions output an Item type and Drawer type. The Find item action
outputs an Item type.
The Hardware Funtionality test action outputs a function call. And, lastly the Activate Listen Mode action outputs a function call.
High
IM2 Voice Command Item type containing value of recognized voice command as the name of
the item. High
Table 3-10 Multimodal User Interface Layer Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 25 Test Items
3.6.2 Central Processing Layer Tests
Test ID Input Expected Output/Action Risk
IC1 Item object with valid name to Item Look-Up
Controller Boolean value representing if item found,
Drawer list sent to HCG-Command Handler High
IC2 Item object with valid name and Drawer object to
add into Boolean value representing if item added to
database Medium
IC3 Item object with valid name and Drawer object to
remove from Boolean value representing if item removed
from database Medium
IC4 Function call with no parameters to Item Loader List of Drawer objects containing lists of Item
objects High
IC5 Function call with no parameters to Indicator Tester Call HCG-Command Handler to activate all LED’s Low
Table 3-11 Central Processing Layer Tests
3.6.3 Storage Layer Tests
Test ID Input Expected Output/Action Risk
IS1 Item object with valid name and Drawer
object to add into Boolean value representing if item added to database Medium
IS2 Item object with valid name and Drawer
object to remove from Boolean value representing if item removed from
database Medium
IS3 Function call with no parameters to Item
Loader List of Drawer objects containing lists of Item objects High
IS4 Item object with valid name List of Drawer objects High
Table 3-12 Storage Layer Tests
3.6.4 Indication Layer Tests
Test ID Input Expected Output/Action Risk
II1 Received correct drawer(s) along with their
respective columns and rows
Map Drawer rows to correct colors and prepare the command string to be passed down the rest of the
hardware High
II2 Create a successful connection to the Arduino
controller Display connection status when request fails High
Table 3-13 Indication Layer Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 26 Test Items
3.7 System Validation Tests
Test ID System Test Input Expected Output/Action Risk
SV1 Activate the Supply
Fetch program
User selects (clicks) the large microphone button to enable
the listening mode
The icon/button changes to indicate the program is listening for speech input
High
SV2 Deciphering of the User’s Vocal Input
The user’s vocal input of an item through the microphone
The program properly accepts speech input and once the user speaks an item’s name, a
message is displayed showing the deciphered speech input (word or phrase)
If the input was inaudible, a message is displayed informing the user to speak again
or use the GUI input method
High
SV3 Find the Item Requested
by the User (vocal or GUI input)
Once the program has been activated, the program
properly accepts speech input or GUI input for a requested
item
A message is shown stating whether or not a positive match was found for that item
The correct column and hue of the LED(s) is illuminated
High
SV4 Update Item Database
The user utilizes the GUI to add, remove, or move item
locations in the database/drawers
A confirmation message is shown to the user and the current database locations of
the items are reflected in the GUI High
SV5 Test All LEDs Functionality
The user utilizes the ‘Test All LEDs’ feature/button in the
GUI
All BlinkM LEDs are illuminated for a specific amount of time to indicate if a BlinkM LEDs
is physically malfunctioning Medium
Table 3-14 System Validation Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 27 Risks
4 Risks
4.1 Overview
The following section will specify potential risks that may develop as a result of testing the Supply Fetch system. Each risk can have a specific impact on the system at a level of high, medium or low, depending on the severity. The strategy for each risk will describe a possible approach for eliminating or reducing the risk.
4.2 Risk Table
ID Risk Impact Severity Strategy
1 Fixing bugs creates
new bugs Creates delays for end product High
Find root cause for original bug if possible
2 Damaging hardware
components
Mainly LED components which are quite costly to replace in large
quantities Medium
Verify wiring and setup is correct prior to running full system
3
Logic errors may be missed (condition
paths, unusual scenarios)
System may not function correctly or handle the error appropriately
Medium Identify all possible scenarios
users may be exposed to during run time
4 Static Analysis tools fail to find bugs in
code
Increase in time needed for examining potential flaws in code
Medium Identify false negatives and false
positives
5 Unpredictable
behavior on software or hardware units
Creates delays for end product Low Test at the lowest level as much as
possible prior to integration and progressing to next test level
6 Layer dependencies
found in code
Will need to revisit architecture design if critical dependencies are
found Low
Verify design during implementation
7 Issues from user testing system
prototype
May question critical features or usability if not intuitive to the user
Medium Provide a prototype to customer a
few weeks before deadline
Table 4-1 Risk Table
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 28 Features To Be Tested
5 Features To Be Tested
5.1 Overview
The following features derive from the original requirements and will be tested to ensure that the Supply Fetch System satisfies functional and nonfunctional specifications. The descriptions explain details about how the system will implement the feature or how it affects the user or system when the feature is triggered. There are three levels of risks that will represent the user’s ability to test to their satisfaction:
Low: The feature has been tested; it is fully functional and works as described in the SRS.
Medium: The feature has been tested; however, there may be certain cases where the particular function is not guaranteed to work, and therefore the system will take a best-effort approach.
High: The feature could not be implemented, tested, or be able work appropriately.
5.2 Customer Requirements
5.2.1 Simple User Interface
Risk: Low
Description: With this feature, the main user should primarily interact with at most a single button to search for an item when making the request via microphone or 2 buttons when making the request via the GUI. The administrative user will be able to interact with the system by modifying the items in the shelf as well as their locations.
Test Approach: We will test to verify that all interface components respond appropriately and that the correct events, messages, notifications or errors, occur after being triggered.
5.2.2 System Should Find Item Requested by User
Risk: Low
Description: With this feature, the main user will be indicated, via an LED, the correct location for the item they requested.
Test Approach: We will test to make sure that LEDs respond correctly to user requests; it is however up to the Speech Recognition Engine to verify that the request is mapped to a text value that best represents the speech keyword.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 29 Features To Be Tested
5.3 Performance Requirements
5.3.1 Accented Speech
Risk: Medium
Description: With this feature, users with accents may be able to make requests while their keywords will be matched as best as possible to one of items in the dictionary or grammar. This feature, however, is not always guaranteed to always work as it is only a best-effort functionality .By limiting the model we reduce the chance for the recognizer to pick the wrong word between other similar sounding words in the English dictionary pool.
Test Approach: To test this feature, we will ask for users with accents to participate in item requests to get a sense the accuracy the speech engine can deliver.
5.3.2 Singular vs. Plural Item Request
Risk: Medium
Description: With this feature, users who request items in a plural form will still be able to retrieve them but the functionality, like in the accented speech, may still be limited as cases may exist where the request fails. In most cases the difference between the single and plural versions of a word is by appending the “s” at the end, therefore still having the chance finding the closest match within the dictionary pool.
Test Approach: For the final compiled item list, identify their plural forms and use them in the find-item feature to verify that the requests are still successful.
5.3.3 Background Noise Reduction
Risk: Medium
Description: With this feature, the user will be able to make an item request via microphone without the interference of background noise. Again, this functionality will be delivered as best-effort and will not be guaranteed to work in every case but it is guaranteed that the microphone used for the system uses noise-cancelling circuitry to filter out background noise.
Test Approach: We will try to make test request during noisy environments to verify that the system can still understand the item being asked any user.
5.3.4 Voice Detection Wait Time
Risk: Low
Description: With this feature, the user will have a limited amount of time to speak and verbally state the item they are searching within the office shelf. If within the allowed time range for searching an item, the user completes a phrase, then the system should try to recognize the voice input even if there is still wait time left. Any requests not
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 30 Features To Be Tested
completed within the time range will most likely not process correctly therefore attempting in best-effort to understand the incomplete request. If the user does not state anything within the time interval the system will not capture input therefore asking the user to state their request.
Test Approach: Time the wait time and verify that the events occur at the appropriate cases described for this feature.
5.3.5 Indication Hold Time
Risk: Low
Description: With this feature, the user will have sufficient time to note the item being indicated by the system via an LED. There will be enough time for the user to spot the light and recognize the color so that it can be traced to a row.
Test Approach: Verify that the hold time lasts close enough to the expected wait time being programmed.
5.3.6 Item Request Failure
Risk: Medium
Description: With this feature, if the item fails to be recognized, a verification procedure will execute to ask the user if they really meant the item understood by the system. If the system fails to understand or present valid suggestions then the user will always have the alternative of selecting the item via an interface list from the program.
Test Approach: Provide the system with misunderstanding requests to validate the actions and suggestions provided by the system.
5.4 Maintenance and Support Requirements
5.4.1 Testing Feature for the Indicators
Risk: Low
Description: With this feature, the users themselves can be able to test the working state of each LED in the system via the user interface under a tolls and configuration menu.
Test Approach: We will verify that the test functionality is of the form of “test all” or “test individual” depending on the current that can be provided by the Arduino to each LED.; there may not be enough current to enable all LEDs.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 31 Features To Be Tested
5.5 Other Requirements
5.5.1 Environment Deployment
Risk: Low
Description: With this feature, the system will be able to be deployed in a shelf that is within the allowed dimensions (4 rows x 11 columns). When the system is deployed the system should prompt the users for the dimensions in their workspace.
Test Approach: Verify that the systems management actions and request actions still work appropriately when dimensions are changed. Changing dimensions may affect item mapping therefore we must be careful that this does not confuse the user when making requests.
5.5.2 Item Frequency Tracker
Risk: High
Description: With this feature, the user will be able to track the frequency of items within a period of time. Other miscellaneous data such as most/least requested or most frequent request may be available in a report to the user. This feature, however, will not be apriority and will be implemented only if there is still time available for enhancements. Test Approach: We will verify that the correct counts and statistics reflect the data that the database holds.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 32 Features Not To Be Tested
6 Features Not To Be Tested
6.1 Overview
The following features will not be tested since they are verified by design. In essence, they describe properties of the product and hold minimum or no functionality at all.
6.2 Customer Requirements
6.2.1 Size of Supply Fetch
By default, the size of the system should fall under the specific dimensions. By visual inspection, the dimension limit can be observed, however, our LED aspect of the project is somewhat unique from others in that it will need to compensate for greater lengths to the purpose it holds for finding items.
6.2.2 Prepping of the system
By default, the system has the property of not requiring voice training or preparations such as in training software. Since the system listens for key items and not phrases, it will be limited to only identifying those set of words in the grammar database.
6.2.3 Speech Processing System
By default, the system will have an attached microphone that will listen to user input as after listening mode has been enabled. This feature can be verified visually in the final delivered product.
6.3 Packaging Requirements
All items included in the final package will be verified visually by the user and project members to make sure that no items were left out. The list in this section of the SRS will be checked off in the final product phase.
6.4 Safety Requirements
Safety items are only system options and physical properties built to ensure that the product is not harmful to the user or to make sure that there is no tampering of the system.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 33 Features Not To Be Tested
6.5 Maintenance and Support Requirements
Other than the testing functionality embedded within the application, no further testing will be required because most of the requirements within this section are only information or support artifacts for basic troubleshooting of the system.
6.6 Other Requirements
6.6.1 On/Off Indication
By default, the on/off feature for the system can be visually verified by the user. The laptop device itself can be turned on or off and the microphone has an on/off button as well.
6.6.2 On/Off Voice Detection
By default, it should appear apparent from the GUI that it is waiting for user input whenever the search session has been request via the search button.
6.6.3 Power Supply
By default, the power supply will be attached to each hardware component requiring a power so it will be up to the user to understand from, the system guide, what components need it and that chargers are in a working state.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 34 Overall Test Strategy
7 Overall Test Strategy
7.1 Overview
The Supply Fetch system needs to be tested to ensure it meets the requirements set forth in the System Requirements Specification and to verify that its construction and operation is consistent with the architecture specified in the Architectural Design Specification and Detailed Design Specification documents. This section covers and explains the overall testing strategy that Team Easy Office Solutions will use to test the Supply Fetch prototype.
7.2 Configurations
Configurations of the Supply Fetch system will be tested in order of priority. For example, highest-priority configurations will be tested first, while lowest-priority configurations will be tested last. Some configurations may not be tested. Any configurations concerning future items will not be tested because future items will not be implemented prior to testing.
7.3 Strategy
Testing of the Supply Fetch prototype shall occur in phases. Each phase of testing will include a set of test items that must be tested against the requirements defined in the System Requirements Specification and the architecture defined in the Architectural Design Specification and Detailed Design Specification. The outcome of each test will be recorded as simply as either a pass or fail. The Test Deliverables section details what information is recorded when a test passes or fails. A phase of testing must be complete before the team can begin the next phase of testing. Order of test phases are identical to the order of tests defined in the Test Items sections and are as follows:
1. Hardware Testing Phase 2. Unit Testing Phase
MUI Layer Tests
Central Processing Layer Tests
Storage Layer Tests
Indication Layer Tests 3. Component Testing Phase
MUI Layer Tests
Central Processing Layer Tests
Storage Layer Tests
Indication Layer Tests
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 35 Overall Test Strategy
4. Integration Testing Phase
MUI Layer Tests
Central Processing Layer Tests
Storage Layer Tests
Indication Layer Tests 5. System Validation Testing Phase
Note that, during the Unit, Component, and Integration Testing Phases, each of the Layer Tests can take place independently/concurrently.
7.4 Metrics
Each test’s priority will be determined by metrics defined by Easy Office Solutions. The priority of each test will be ranked according to the requirements defined in the System Requirements Specification document and its effect on the system’s ability to find and notify, to a user, the location of an item in a supply room. Each test will carry a rank of high, medium, or low.
7.4.1 High
Should this test item fail, the system’s overall ability to find and notify the location of an item to a user will be compromised. The failure of this test item also means that one or more high-level requirements in the System Requirements Specification were not met.
7.4.2 Medium
Should this test item fail, the system’s ability to allow a user to perform maintenance or edit data will be compromised. The failure of this test item also means that one or more medium-level requirements in the System Requirements Specification were not met.
7.4.3 Low
Should this test item fail, the system’s ability to allow a user to test part of the system will be compromised. The failure of this test item also means that one or more low-level requirements in the System Requirements Specification were not met.
7.5 Regression
Regression testing will be used by Team Easy Office Solutions during the development of Supply Fetch. This testing will occur every time a bug or malfunction is found and every time a new component is integrated into Supply Fetch. Hardware pieces, modules, subsystems, and layers are all considered components. During testing, it’s very likely that a bug or malfunction of the system will be encountered, causing the test to fail. When we fix the bug or malfunction, any components that were involved in the test failure will be put through regression testing. All original tests performed prior to the bug or malfunction will be repeated in order to make sure that all of the components tested so far are still working correctly. Once all of the tested components successfully pass regression testing, normal testing of the system’s components can resume.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 36 Acceptance Criteria
8 Acceptance Criteria
8.1 Overview
The Acceptance (Pass/Fail) Criteria below will be used to determine if a test on a particular aspect of the Supply Fetch project yielded a positive (pass) or negative (fail) result. The tests have been divided into five different categories: hardware, module, subsystem, layer, and system verification testing.
8.2 Hardware Testing
The given inputs and the expected outputs for Hardware Testing are based off documentation in the DDS and manufacturer documents (if applicable).
Pass The individual hardware unit returns the correct/expected output when given a valid input. The individual hardware unit returns an error or exception when given an invalid input.
Fail The individual hardware unit returns incorrect/unexpected output when given a valid input. The individual hardware unit does not return an error or exception when given an invalid input.
8.3 Unit (Module) Testing
The given inputs and the expected outputs for Unit Testing are based off documentation in the DDS document.
Pass The module returns the correct/expected output when given a valid input. The module returns an error or exception when given an invalid input.
Fail The module returns incorrect/unexpected output when given a valid input. The module does not return an error or exception when given an invalid input.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 37 Acceptance Criteria
8.4 Component (Subsystem) Testing
After every module within a subsystem has passed Unit Testing, that subsystem as a whole will be tested. The given inputs and the expected outputs for Component Testing are based off documentation in the ADS and DDS documents.
Pass The subsystem returns the correct/expected output when given a valid input. The subsystem returns an error or exception when given an invalid input.
Fail The subsystem returns incorrect/unexpected output when given a valid input. The subsystem does not return an error or exception when given an invalid input.
8.5 Integration (Layer) Testing
After every subsystem within a layer has passed Component Testing, that layer as a whole will be tested. The given inputs and the expected outputs for Integration Testing are based off documentation in the SRS, ADS, and DDS documents.
Pass The layer returns the correct/expected output when given a valid input. The layer returns an error or exception when given an invalid input.
Fail The layer returns incorrect/unexpected output when given a valid input. The layer does not return an error or exception when given an invalid input.
8.6 System Verification Testing
After every layer within the system has passed Integration Testing, the entire Supply Fetch system as a whole will be tested. The given inputs and the expected outputs for System Verification Testing are based off documentation in the SRS, ADS, and DDS documents.
Pass The whole system returns the correct/expected output when given a valid input and fulfills/meets all relevant requirements defined in the SRS document. The whole system returns an error or exception when given an invalid input.
Fail The whole system returns incorrect/unexpected output when given a valid input or does not fulfill/meet all relevant requirements defined in the SRS document. The whole system does not return an error or exception when given an invalid input.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 38 Test Deliverables
9 Test Deliverables
9.1 Overview
This section will cover all documentation, deliverables, and other materials that will be provided with the end project documentation.
9.2 Deliverables
9.2.1 System Test Plan
This document will provide the necessary related material to fully describe everything within the System Test Plan.
9.2.2 Test Case Specification
This deliverable will include
General Description of component that is focus of test case.
Valid range of inputs expected for correctly working component.
Valid range of outputs to be expected from correctly working component.
9.2.3 Test Results
This deliverable will include
Test ID
Date of Test
Tester Name
Inputs
Expected Outputs
Actual Outputs
Result of Test (Pass/Fail)
Tester Notes
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 39 Test Deliverables
9.2.4 Bugs and Defects
This deliverable will include all documentation that covers the case of a Fail result, to fully cover this information it will include.
Test ID
Date of Test
Tester’s Name
Description of Test Failure
Status (Fixed/Ignored)
Severity High
Overall feature of finding an item is compromised.
At least one high-level requirement not met. Medium
Maintenance features (adding/removing items) compromised.
At least one medium-level requirement not met. Low
Testing features (test all LED’s) compromised.
At least one low-level requirement not met.
Effects on development
Effects on testing
Other
9.2.5 Test Code
This deliverable will include all code used for testing in the end product documentation provided with the prototype to the customer.
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 40 Test Schedule
10 Test Schedule
10.1 Overview
A schedule of the testing process for the Supply Fetch product is included in section 10.2. This schedule is consistent with the testing tasks in our Project Plan document.
10.2 Test Schedule
Project Plan Task Number
Task Planned Start Planned Finish
2.6.2.1.1 Hardware Testing 3/29/2013 3/31/2013
2.6.2.2.1 MUI Layer Unit Tests 3/31/2013 4/3/2013
2.6.2.2.2 Central Processing Layer Unit Tests 3/31/2013 4/3/2013
2.6.2.2.3 Storage Layer Unit Tests 3/31/2013 4/3/2013
2.6.2.2.4 Indication Layer Unit Tests 3/31/2013 4/3/2013
2.6.2.3.1 MUI Layer Component Tests 4/3/2013 4/6/2013
2.6.2.3.2 Central Processing Component Tests 4/3/2013 4/6/2013
2.6.2.3.3 Storage Layer Component Tests 4/3/2013 4/6/2013
2.6.2.3.4 Indication Layer Component Tests 4/3/2013 4/6/2013
2.6.2.4.1 MUI Layer Integration Tests 4/6/2013 4/10/2013
2.6.2.4.2 Central Processing Layer Integration Tests 4/6/2013 4/10/2013
2.6.2.4.3 Storage Layer Integration Tests 4/6/2013 4/10/2013
2.6.2.4.4 Indication Layer Integration Tests 4/6/2013 4/10/2013
2.6.2.5.1 System Validation Tests 4/10/2013 4/16/2013
Table 10-1 Test Schedule
System Test Plan Project Name: Supply Fetch
Easy Office Solutions 41 Approvals
11 Approvals
The section provides the approval of this document’s presented plan by all necessary individuals to the project.
Name Signature Date
Mike O’Dell
Darin Brezeale
Nicholas Burns
Shawn Gilleland
Miguel Sanchez
Gideon Miller