Certification Test Tool Update
Toby NixonProgram Manager
Microsoft Corporation
Assumptions
Familiar with UPnP™ v1 architecture and terminology
Interested in evaluating a device’s compliance against the UPnP™ standard
Pursuing the UPnP™ Logo
Overview Certification Test Tool purpose What is tested Test capabilities Using the tool and interpreting results Update on the next release
Purpose of Cert Tool
Drive the certification of UPnP™ devices Execute device independent protocol
tests against the UPnP™ v1 standard Support device dependent syntax and
semantic testing
What is Tested
Device-independent protocol tests Device-specific syntax tests Security Device tests Device-specific semantic tests
Protocol Tests
Validate SSDP and HTTP packets and their sequence
Test the following stages: Addressing, Description, Discovery, Security, Control, Eventing
Execute regardless of device type
Syntax Tests Actions to be tested for each device type are
specified in an XML configuration file Syntax tests are created by editing the
<ServiceId>.SyntaxTests.XML file Working groups provide the XML syntax test
configuration files Tests are run during the Control and
Eventing test stages Actions are invoked in the order in which
they appear in the XML file
Device Security Tests
Available in test tool versions 1.4 and above
Device Security tests as part of the service syntax tests (3.*)
The test configuration file is ‘DeviceSecurity1.SyntaxTests.xml’
Semantic Tests Verify specified output parameters, error
conditions and events Create semantic tests by editing the
<ServiceId>.SemanticTests.XML file Working groups provide the XML syntax test
configuration files Actions must be within the same service Results of one action cannot be used as
input parameters in another action
What is Not Tested Does not test performance specifications Does not certify device extensions external
to the UPnP™ standard Does not test incorrect inputs, boundary
conditions Does not test product capabilities
Tests on-the-wire interoperability
Test Environment Test tool requires Windows XP
Recommend XP SP2 (for IPv6 and HTTP/1.1 testing)
Run only one UPnP™ device and a single instance of the test tool on an isolated network
Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP
Test Environment Diagram
Isolated Isolated Network HubNetwork Hub
Cert Tool running on WinXPCert Tool running on WinXPDevice Under TestDevice Under Test
DHCP Server for DHCP Server for Addressing testsAddressing tests
Network Sniffer Network Sniffer (optional)(optional)
XML Configuration Files <DeviceId>.xml
Lists the implemented services and embedded devices
<ServiceId>.xml Lists the implemented Actions and State Variables
<ServiceId>.SyntaxTests.xml Lists the Action names and the ‘In’ arguments for each Action
<ServiceId>.SemanticTests.xml Lists the Action names, ‘In’ arguments and the expected ‘out’
arguments or error code.
Using XML Config Files Working Committees
Write the XML config files for standardized DCPs Distribute XML config files on the UIC web site
The test tool uses the XML files to verify Devices have all required services Services implement required Actions and State
Variables Devices respond correctly for each Action
invocation Proprietary devices can be tested but require
the respective XML config files
Device Description<device>
<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>
</device><device>
<Optional/> <deviceType>urn:schemas-upnp-
org:device:LANDevice:1</deviceType></device>
Service Description<serviceStateTable> <stateVariable> <name>EthernetLinkStatus</name> <dataType>string</dataType> <allowedValueList> <allowedValue>Up</allowedValue> <allowedValue>Down</allowedValue> </allowedValueList> </stateVariable></serviceStateTable>
Syntax Tests XML File<TestCase>
<Id>1</Id><Category>Valid Action And Valid InArgs</Category><ActionName>GetEthernetLinkStatus</ActionName>
<ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode>
</TestCase>
Semantic Tests XML File<ManualAction>
<ActionInstruction> Remove cable. </ActionInstruction><VerificationInstruction> Verify link light is off.
</VerificationInstruction><ExpectedSuccessResponse/>
</ManualAction>
Test Tool UI
Select Test Options
Step Mode / Run Until Failure Step Mode
Pauses between each test Click the “Next” button to continue Useful when debugging a failure
Run Until Failure Pauses the test execution at the first failure
Interpreting the Test LogTest Case 2.0.0.1 PassedTest Case 2.1.0 Failed to RunFailure Reason :Precondition Failure - Unable to
Construct Device Discovery Message List.Reason : Unable to Acquire Root Device Description
Url.Badly formed Notify Alive Packets Header Value.
Reason : NT Header Value = TestDevice is Not Compliant to UDA V1.0
Helpful Hints Review the test log for failure details Use a network sniffer
UPnP™ parser available for Microsoft Network Monitor
Useful for reporting problems Use Step Mode to help isolate an issue Refer to the Test Tool User documentation
for descriptions of each test case
Helpful Hints Continued A full certification pass takes about 25
minutes to complete For testing purposes, set the CACHE-
CONTROL in the device to be less than 60 for SSDP alive notifications
In your final device implementation, set the CACHE-CONTROL according to UDA recommendations
Update on next release v1.5 Planned for 2nd Quarter 2005 New features
Using MSEARCH to initiate tests instead of NOTIFY for faster test runs
Improved logging messages for better diagnostics Testing on non isolated networks and presence of
multiple root devices Embedded devices testing Comparing arguments and passing parameters
across different actions Presence of extra header information in Discovery,
Description, Control and Eventing tests
Q&A Certification Test Tool
Is made available by UIC Tool is posted on www.upnp-ic.org Documentation is included with the tool For debugging, Cert Tool in UPnP™ Forum website has
sources Support
UIC members receive one free support call Newsgroup: microsoft.public.upnp
Questions Email: [email protected]
Foundation of the Connected Home