+ All Categories
Home > Documents > 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the...

2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the...

Date post: 13-May-2020
Category:
Upload: others
View: 5 times
Download: 0 times
Share this document with a friend
58
System Verification Test Plan For Diamond R2 Application Phase A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Transcript
Page 1: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

System Verification Test PlanFor

Diamond R2 Application Phase A

1

2

3

4

5

67

8

9

10

11

12

13

14

15

16

17

18

19

Page 2: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

TABLE OF CONTENTS

1 INTRODUCTION..................................................................................................................................5

1.1 IDENTIFIER........................................................................................................................................51.2 OBJECTIVES.......................................................................................................................................51.3 BACKGROUND...................................................................................................................................51.4 GOALS...............................................................................................................................................61.5 REFERENCES.....................................................................................................................................61.6 BIBLIOGRAPHY..................................................................................................................................71.7 ACRONYMS.......................................................................................................................................7

2 TEST ITEMS..........................................................................................................................................8

2.1 PRODUCT LEVEL MODULES..............................................................................................................82.2 REQUIREMENTS SPECIFICATION REFERENCE....................................................................................82.3 DESIGN SPECIFICATION REFERENCE.................................................................................................82.4 USERS GUIDE REFERENCE................................................................................................................82.5 INSTALLATION GUIDE REFERENCE...................................................................................................8

3 TEST ACTIVITY...................................................................................................................................9

3.1 AREAS OF TESTING...........................................................................................................................93.1.1 Diamond Core Testing.............................................................................................................93.1.2 Diamond Application Testing...................................................................................................9

3.2 FUNCTIONAL INTEGRATION TESTING.............................................................................................103.2.1 Integration Test......................................................................................................................103.2.2 Full Coverage Test.................................................................................................................103.2.3 Regression Test.......................................................................................................................10

3.3 TEST ENVIRONMENT SET-UP..........................................................................................................113.3.1 Automated Testing Configuration..........................................................................................113.3.2 Manual Testing Configuration...............................................................................................12

4 FEATURES TO BE TESTED.............................................................................................................14

4.1 DIAMOND CORE TESTING.................................................................................................................144.1.1 Core Requirements.................................................................................................................144.1.2 Adapter Framework Requirements.........................................................................................154.1.3 SOAP Gateway Requirements................................................................................................174.1.4 SIP Adapter Requirements.....................................................................................................204.1.5 Management Services Requirements (new header)................................................................214.1.6 Performance Requirements....................................................................................................214.1.7 Software Platform...................................................................................................................224.1.8 Lifecycle Management............................................................................................................234.1.9 Remote Access........................................................................................................................234.1.10 Secure Access.........................................................................................................................234.1.11 Mandatory Requirements.......................................................................................................24

4.2 DIAMOND APPLICATION TESTING...................................................................................................274.2.1 Click-To-Call Web Service (Diamond SRAD)........................................................................274.2.2 B2BUA Interface....................................................................................................................284.2.3 Deployment Models................................................................................................................294.2.4 Installation and Upgrades......................................................................................................29

5 FEATURES NOT TO BE TESTED BY SV.......................................................................................33

5.1 CRYSTAL...........................................................................................................................................335.2 SES..................................................................................................................................................335.3 CM..................................................................................................................................................33

20

2122

23

24252627282930

31

3233343536

37

38394041424344454647

48

4950515253545556575859606162636465

66

676869

Page 3: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

5.4 DIAMOND CORE TESTING.................................................................................................................335.4.1 [113240-501]: Standard Axis Deployment...........................................................................335.4.2 [113240-400]: Vendor Independent JBI Integration............................................................33

5.5 DIAMOND APPLICATION TESTING.....................................................................................................335.5.1 [113358-2160] Login Session Properties..............................................................................335.5.2 [113358-1840] Databases (postgres and LDAP) should be preserved across upgrades......34

6 TEST STRATEGY / APPROACH.....................................................................................................35

6.1 STAGE 1 (SETUP)............................................................................................................................356.2 STAGE 2 (100% COVERAGE TEST CYCLE).....................................................................................356.3 STAGE 3 (REGRESSION TEST CYCLE).............................................................................................356.4 ENTRANCE CRITERIA........................................................................................................................36

6.4.1 Stage 1 (Setup)........................................................................................................................366.4.2 Stage 2 (100% Coverage).......................................................................................................366.4.3 Stage 3 (Regression)...............................................................................................................36

6.5 EXIT CRITERIA.................................................................................................................................366.5.1 Stage 2 (100% Coverage).......................................................................................................366.5.2 Stage 3 (Regression)...............................................................................................................36

7 PASS / FAIL CRITERIA.....................................................................................................................37

7.1 BUG ANALYSIS / REPORTING..........................................................................................................377.2 QUALITY METRICS..........................................................................................................................37

8 SUSPENSION CRITERIA AND RESUMPTION REQUIREMENTS..........................................38

8.1 SUSPENSION CRITERIA....................................................................................................................388.2 RESUMPTION REQUIREMENTS.........................................................................................................38

9 TEST DELIVERABLE........................................................................................................................39

9.1 TEST DESIGN SPECIFICATIONS.........................................................................................................39

10 TESTING TASKS / MILESTONES...............................................................................................40

10.1 TESTING TASKS...............................................................................................................................4010.2 MILESTONES...................................................................................................................................40

11 ENVIRONMENTAL NEEDS.........................................................................................................41

11.1 HARDWARE.....................................................................................................................................4111.2 SOFTWARE......................................................................................................................................4111.3 OTHER DEPENDENCIES...................................................................................................................41

12 RESPONSIBILITIES.......................................................................................................................41

13 STAFFING AND TRAINING.........................................................................................................42

13.1 STAFFING........................................................................................................................................4213.2 TRAINING........................................................................................................................................42

14 SCHEDULE......................................................................................................................................43

14.1 SCHEDULE.......................................................................................................................................4314.1.1 Stage 1 - Setup........................................................................................................................4314.1.2 Stage 2 – 100% Coverage......................................................................................................4314.1.3 Stage 3 – Regression..............................................................................................................44

15 RISKS AND CONTINGENCIES...................................................................................................45

15.1 RISKS...............................................................................................................................................4515.2 CONTINGENCIES..............................................................................................................................45

16 REVIEWS AND APPROVALS......................................................................................................46

707172737475

76

77787980818283848586

87

8889

90

9192

93

94

95

9697

98

99100101

102

103

104105

106

107108109110

111

112113

114

Page 4: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

16.1 DOCUMENT APPROVAL SIGN OFF...................................................................................................46

17 REVISION HISTORY.....................................................................................................................47

18 APPENDICES...................................................................................................................................48

18.1 Appendix 1.....................................................................................................................................48

115

116

117

118119

Page 5: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

1 INTRODUCTIONThis master test plan describes the various testing activities and responsibilities that are planned for System Verification on the Diamond R2 phase A product. This document is compliant with IEEE 829 standards.

1.1 Identifier

The SV conferencing group maintains this document, SV Test Plan for Diamond R2 Application Phase A. This document will be updated as needed.

Test Plan Identifier : Diamond R2 Phase A SLTPCompas ID : 120139

1.2 Objectives

The objectives of the test plan include

Detail the activities required to prepare and perform system level testing for the product. To communicate with the respective responsible parties those activities that they are to

perform and schedule to be followed. To communicate to suppliers & customers of the project test team the dependencies, and

external requirements of the project. To define the tools, technology, and environment needed to deliver testing within the

Avaya conferencing SV group.

1.3 Background

Avaya has traditionally managed products and application enablement offerings focused to the individual product needs. When solution level offerings arise, pairwise integration occurs for that specific solution. The end result is a series of nonuniform APIs, difficult integration in the portfolio, and inconsistencies in many aspects of the portfolio presented to our customers.

At the same time, customers have been looking to streamline their business applications and business processes through consistent user experience, consistent integration, and easy to design workflows that meet their business needs. While a lot of this space is out of scope for Avaya, there is one key aspect that Avaya can add a lot of value to customers. This value is in communication enabling business applications and business processes. For example, inside a supply chain workflow, the customer sees a lot of value in adding the ability to initiate communications based on customer supplied business logic and rules.

Diamond is a new approach to offering Avaya’s customers a form of application enablement that is agile enough to adapt to different customer environments on one end, and be supported by the portfolio of communications infrastructure Avaya offers on the other end. To this end, Diamond can be viewed both as a offer that can be easily deployed for customers that need the specific set of capabilities, and at the same time, as a flexible professional services offering where Avaya builds a tailored system to meet those customers’ requirements.

120121122123124

125

126127128129130131

132

133134135136137138139140141142

143

144145146147148149150151152153154155156157158159160161162163

Page 6: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

The Diamond product is essentially a software offering for high level application enablement offerings that can very easily be adapted to different customer environments and to different Avaya infrastructure. The first set of Diamond services available are click-to-call (including click-to-conference), exception based conferencing, and notification / response services al 17 l powered by Avaya’s rich voice and multimedia infrastructure. These services are used by customers to communications enable their business applications. For example, a supply chain application may encounter an exception and today it just sits there and hangs around until someone sees the alarms. Using Diamond, this application invokes the exception conference application and sends it the data to get the correct people into a conference.

To accommodate the just-in-time response to customer environments and proposals, and to the large spectrum of Avaya infrastructure and Avaya customers available, the Diamond architecture is both service-oriented and adaptable to insure the possibility of composable offerings. The end result is a base software ecosystem that has several upgrade paths for the customer to enhance their features with additional third party software. Each Diamond service participates in an internal service oriented environment where feature churn and feature development cost is reduced via service discovery, service orchestration, service description and meta-modeling using standards based technology.

The Diamond R2 Application is an iterative release that addresses this need by building up a suite of web services for communications enablement of business processes and applications. With the iterations (called phases in this document), the customer is presented with additional capabilities, greater return on investment, and a richer set of target applications. This document includes the requirements for Phase A (the first iteration) which provides a web services interface for click-to-call (including click-to-conference) and Phase B (the second iteration) which provides a customizable platform including reference implementations of a number of communications-related web services. Both phases are built on the Diamond Core infrastructure described in CID 113240.

1.4 Goals

The goals of the SV Conferencing Team for Diamond R2 Phase A are: Ensure that the Diamond R2 Phase A solution meets all quality requirements during each

test. Qualify all features Isolate Functional Problems Isolate Performance Problems Perform Regression tests Perform load and stress testing Isolate Security Problems Perform bug fix verification Review Documentation

1.5 References

Document Name Compas IDSRAD: Diamond Core 113240SRAD: Diamond R2 Application 113358VIA System SRAD 117418Diamond R2 PRD 112824

164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193

194

195196197198199200201202203204205206

207

Page 7: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

Crystal Release 2 SRAD 114190Installation and Configuration Framework: RFS

107559

Core Services Phase 2 – Secure Data Exchange SRAD

109144

1.6 Bibliography

Document Name Compas ID

1.7 Acronyms

The Diamond terminology is defined in issue 1.0 of the Diamond Terminology document, CID 114625. The GCS Dictionary, CID 110276, may also be a useful reference. In addition these terms may also be found in this document:

TERM MEANINGATF Avaya Test FrameworkJBI Java Business IntegrationNMR Normalized Message Response

208

209

210

211

212213214215

216217

Page 8: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

2 TEST ITEMSThe products covered by this plan are listed in section 2.1

2.1 Product Level Modules

For Diamond R2 Phase A the following modules will be tested:

Click-to-* web service SOAP Router B2BUA Adaptor

2.2 Requirements Specification Reference

PRD Diamond Release 2 Draft Issue 1.0.1 – Compas ID: 112824Dated: Oct 10th 2005

2.3 Design Specification Reference

SRAD Diamond R2 Application Issue 2.0 – Compas ID: 113358Dated: Apr 12th 2006

2.4 Users Guide Reference

User Guide for Diamond 2.0 – Compas ID: 117840Dated: No Date (N/A at time of writing)

2.5 Installation Guide Reference

Reference not found in Compas

218219220

221

222223224225226227

228

229230

231

232233

234

235236

237

238239240

Page 9: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

3 TEST ACTIVITY

3.1 Areas of TestingThe test areas described in this document focus on testing the Diamond Core Services functionality and the Diamond R2 Application, namely click-to-* functionality of Diamond R2. Diamond R2 Phase A introduces most components of the Diamond Core (CID 113240) and a single web service for customer use allowing an n-party SIP call to be established using an Avaya Meeting Exchange S6100 (Crystal).

3.1.1 Diamond Core Testing

Phase A includes the following basic infrastructure components 1. JBI enterprise service bus from Active Queue / ServiceMix2. The adaptor framework from ServiceMix3. Normalized message router 4. SOAP Gateway provided by 2SAP with Apache Axis5. SIP B2BUA using Flextronics SSF to provide the n-party click to call

Testing of the Core Requirements will cover Installation and Functionality of each component, Security and Performance.

3.1.2 Diamond Application Testing

The Diamond R2 Phase A application described in this document is the first application built using this core, and consists of a web service providing a generalized click-to-call for one or more parties using Avaya’s Crystal R1.5 conference bridge.

Testing of the Diamond Application:1) Testing the web service to place a SIP call to one or more destinations from an originator. (Click-to-Call)

241

242243244245246247248

249

250251252253254255256257258

259

260261262263264265266

Page 10: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

3.2 Functional Integration Testing

3.2.1 Integration Test

This outlines an initial set of test cases which can be used to perform end-to-end testing on Diamond.

The following core services will be available on the platform and must be tested: Core Requirements Adapter Framework Requirements

Integration testing will focus on system configuration and usability in the following components:

Verify the JBI Enterprise Service Bus SOAP Gateway Requirements SIP Adapter Requirements Management Services Requirements Performance Requirements Software Platform Lifecycle Management Remote Access Secure Access Mandatory Requirements

The following reference service will be available on the platform and must be tested:

Click-To-Call Web Service (Diamond SRAD) B2BUA Interface Deployment Models Installation and Upgrades Diamond R2 Operation, Administration, and Maintenance

These derived test cases will cover aspects of validating the above services. For the most part where possible, automated testing will be performed. This means for the most part, the recipients that are being notified by the above stations will be devices (CMAPI stations) that are also controlled by ATF.

3.2.2 Full Coverage Test

Full coverage testing will involve a full cycle of testing to examine click-to-call integration with the platform.

This includes the following Re-running tests performed in integration test phase in order to verify bug fixes. Full Cycle Test of end-to-end tests Testing of Click-to-Call web service Verification of installed components Core security requirements.

3.2.3 Regression Test

Regression testing will consist of a targeted selection of test cases from the full-coverage testing phase. The chosen test cases will depend on risk areas with outstanding bugs in GA candidate builds. As such, a definitive list will only be compiled at the start of the regression phase

1

267

268

269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301

302

303304305306307308309310311

312

313314315316

2

Page 11: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

3.3 Test Environment Set-upThe test environment consists of two configurations.

Automated Testing Configuration Manual Testing Configuration

3.3.1 Automated Testing Configuration

Automated Testing Configuration consists of a Diamond Automation Machine running the ATF automated test cases on a scheduled basis. The automated test cases are reported via email or a web page.

Because the Diamond web service components are called remotely across a network e.g. Click-to-conf, Notify-response, the ATF is located on a separate machine. This is also the case for SIP calls where the B2BUA or Crystal creates a SIP connection across a network. And OA&M where administration web pages must be accessible across a network.The ATF Automation Machine can also run tests against other Diamond Machines by changing configuration files.

Figure 1 – ATF Automation Testing Configuration

3.3.1.1 ATF Automation MachineThe ATF Application is scheduled to run using Cruise Control.

Reporting is created at the time of ATF Application execution by Cruise Control and reports are mailed to the concerned parties.

The ATF SIP Phone is started from within the ATF and acts as a SIP endpoint for testing SIP calls and SIP RFC compliance.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

11

3

317318319320

321

322323324325326327328329330

331332

333334335336

337338339340341342343344

456

Page 12: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Tomcat is the Web Container running on the ATF Automation Machine for making reports available via a web interface.

The ATF Automation Machine also calls Crystal and the Avaya Call Manager to execute its test cases.

3.3.1.2 Diamond Automation Machine

The Diamond Automation Machine is a Diamond machine with the latest version of the Diamond and OA&M applications installed plus the ATF Client Application. It is available at all times for testing from the ATF Automation Machine.

Because the ATF is running on a remote machine which doesn’t have access to local properties on the Diamond Automation Machine, some tests must execute remotely from the ATF Automation Machine e.g. JVM version, OS version, running services.The ATF Client Application is a Java application running within Axis that exposes web services that the ATF Application calls to complete its test scripts.

Example: ATF Application calls a Web Service on the ATF Client Application that returns the JVM version. If the version is equal to 1.5 the ATF script returns a pass else fail.

3.3.1.3 Diamond Automation ATF Testing ToolsIn addition to the ATF client the ATF uses a number of other Java API’s and tools for testing the Diamond application.

Tool or API Example of testsSSHTools (Java API)

Testing protocols are activated such as sshExecuting linux commands

JMX API Version numbers of servicemixTesting components on the JBI bus

HTTPUnit Testing administration web pagesHttp and Https access

Web Service Client (stubs generated with WSDL2Java)

Click to Call Notification serviceAdvisory web service

ATF Client App (Apache axis application)

Version numbers of JVM, OSTesting components on the JBI bus

3.3.2 Manual Testing Configuration

Where automated testing is not possible or viable the Manual Testing Configuration consists of the SQA Engineers ATF machine and a Diamond Machine.

The SQA Engineer executes the ATF script and manually intervenes to complete the test case. E.g. this may involve manually picking up a handset to verify call quality.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

12

7

345346347

349350351

352353354355356357358359360361362363364365366

367

368369

370371

372373374

375

376377378379380381382383

89

10

Page 13: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

13

11

384385

121314

Page 14: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

4 FEATURES TO BE TESTEDTesting of this product will involve verification of the following features:

Diamond Core Testing Diamond Application Testing

All requirement numbers are as defined in the SRAD.

4.1 Diamond Core Testing

Diamond R2 shall provide a Click to Call web service for initiating a call from an originator to one or more called parties. The originator and called parties shall be identified by URIs. The web service call shall be synchronous from the invoking client’s view, returning success if the originator is successfully added to the call. This service will be built upon the Diamond Core infrastructure. The core infrastructure will run on Redhat Enterprise Linux 4.0. Diamond Phase A includes the following basic infrastructure components

6. JBI enterprise service bus from Active Queue / ServiceMix7. The adaptor framework from ServiceMix8. Normalized message router 9. SOAP Gateway provided by 2SAP with Apache Axis10. SIP B2BUA using Flextronics SSF to provide the n-party click to call.

Testing of the Core Requirements will cover from Installation, functionality of each component through to security and performance.

4.1.1 Core Requirements4.1.1.1 [-10]: Default JBI ProviderDiamond R2 shall utilize the ServiceMix 3.0 (or later) JBI Enterprise Service Bus by default in this release.

No specific action is needed for this requirement. ServiceMix will be exercised by other testing.

1. Verify the ServiceMix version via a script after each install and record this information.

4.1.1.2 [-20]: JBI BusThe Diamond Core shall utilize a JBI Enterprise Service Bus for communication between all Diamond services.

Notes: Configuration and initialization/discovery messaging as well as ongoing communication traffic will follow the JBI standard. The use of a bus provides for a loose coupling between the producer and consumer and provides location independence (producers/consumers can be in the same JVM as in Diamond R2, or in later releases in different JVM’s on the same server or different servers).

This will involve verifying the following:1. Verify configuration initialization/discovery messaging as well as ongoing

communication traffic follows the JBI standard.

4.1.1.3 [-40]: JBI Message RoutingDiamond components shall utilize NMR implicit endpoint selection for routing request messages between the various components.

This will involve verifying the following:1. Verify Normalized Messages from the adaptors are place onto the JBI bus.2. Verify Normalized Messages delivered to correct adaptor from the JBI bus.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

14

15

386387388389390391392

393

394395396397398399400401402403404405406407

408409410411412413414415416

417418419420421422423424425426427428429430

431432433434435436437

161718

Page 15: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

4.1.1.4 [-46]: JMS bus not externally accessibleThe JMS Message Bus for Diamond R2 shall not be visible via any external Ethernet interface.This will involve verifying the following:

1. Verifying that JMS Bus is not externally accessible.

4.1.1.5 [-92]: Character set for messagesCharacter data in all messages on the bus shall be passed in UTF-16, which is Java’s default internal character format, with serialization into UTF-8 for non-Java interactions (e.g. XML).This will involve verifying the following:

1. Verify all message on the bus are in UTF-16 character encoding.2. Verify messages sent using non-Java interactions (SOAP/XML) are UTF-8 character

encoded.

4.1.1.6 [-100]: Component Naming ConventionEach bus service shall use a unique name following a URI scheme.

This will involve verifying the following:1. Verify the component names can be retrieved from ServiceMix.2. Verify component names have the format jbi:servicename@diamonddomain.

4.1.1.7 [-200]: JBI registry InterfaceThe JBI container shall provide a registration interface to insert, update, delete and retrieve information about adapters including their supported capabilities, and current state. This will involve verifying the following:

1. Verify information on an adapters capabilities and current state can be retrieved2. Verify information on an adapters capabilities and current state can be deleted 3. Verify information on an adapters capabilities and current state can be inserted4. Verify information on an adapters capabilities and current state can be updated

4.1.2 Adapter Framework Requirements

4.1.2.1 [-300]: ServiceMix 3.0

Diamond R2 shall use JBI, specifically ServiceMix 3.0 (or later), as the basis for the Adapter Framework.

This will involve verifying the following:1. Verify the standard JBI Component Interface is used as the adapter framework.2. Verify the Normalized Message Router is used for routing messages on the bus3. Ensure no non-standard JBI extensions are used.

4.1.2.2 [-302]: Adapter Session ManagementA common adapter library available to all Diamond JBI components shall provide Session Management capabilities for use by Diamond components. The Session ID and originator’s JBI address shall be exposed in a “Diamond header” element in the body of the messages sent and received by the Diamond components. Multiple asynchronous “event” messages may be sent as responses to a single request message.

This will involve verifying the following:1. Verify all Diamond JBI components provide Session Management Capabilities.2. Verify the Session ID is exposed in the header element in the body of all messages

sent and received via Diamond Components.3. Verify the Originator’s JBI address is exposed in the header element in the body of all

messages sent and received via Diamond Components.4. Verify the Session ID is returned in the response message to adaptor that made the

request.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

15

19

438439440441442443444

445446447448449450451452

453454455456457458

459460461462463464465466467

468

469470471472473474475476477478479

480481482483484485486487488489490491492493494

202122

Page 16: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

4.1.2.3 [-310]: Adapter LoggingAll Diamond JBI components shall provide generic logging and tracing of service startup, service shutdown, and all messages sent/received on the bus on behalf of the component.

This will involve verifying the following:1. Verify the Core Services logging Interface is used for all logging activities by diamond

adaptors.2. Verify at least the following information is logged for each adaptor

a. Component startupb. Component shutdownc. Messages sent to the bus by the componentd. Messages received from the bus by the component.

4.1.2.4 [-350]: Adapter Capabilities Exchange and RegistrationDiamond JBI components shall use JBI’s mechanisms for registering the component’s capabilities. ServiceMix supports hot deployment of JBI components.

This will involve verifying the following:1. Verify Diamond JBI components can be deployed when ServiceMix is running2. Verify Diamond components utilise JBI standard mechanism for registering

component capabilities.

4.1.2.5 [-360]: Opaque Context Data reflectionAdapters shall communicate back any opaque context data (message metadata) received from the message bus (such as SessionID, context information, etc.) in responses associated with an inbound request.Notes: The JBI normalized message format includes a variable length section of context data (metadata). Diamond components will return (intact) any unrecognized metadata in any response. This allows the Diamond components to recover suspended activities and to run “stateless” by embedding state information in message context data. Interactions of this mechanism with message flows including the PXE BPEL engine are not currently completely understood. It is possible message metadata cannot be accessed by the PXE BPEL engine, in which case Diamond components should not count on using metadata.

This will involve verifying the following:1. Verify context/metadata can be added to a message and accepted by the adaptor.2. Verify context data is returned in the response to the inbound request.3. Verify the adaptors cope with different size information submitted.

4.1.2.6 [-370]: Adapter integration with JBIAll Diamond JBI components shall interface with other Diamond components using the JBI bus.This will involve verifying the following:

1. Verify all messages between adaptors are over the JBI bus.

4.1.2.7 [-382]: Adapter shutdownAll Diamond JBI components shall support a shutdown MBean allowing the adapter a chance to execute an orderly shutdown.This will involve verifying the following:

1. Verify all Diamond components provide the shutdown capability.2. Verify components shutdown cleanly when requested.3. Verify component shutdown is logged.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

16

23

495496497498499500501502503504505506507508

509510511512513514515516517518

519520521522523524525526527528529530531532533534535536

537538539540541542543

544545546547548549550551

242526

Page 17: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

4.1.2.8 [-390]: Adapter DeploymentDiamond JBI components shall be packaged as a JBI service assemblies; the manifest will contain the “adapter-class”(s) that conforms to the JBI NMR, life cycle, and management interfaces.Notes: Adapters can theoretically be developed in languages other than Java by providing a Java wrapper to be deployed into the framework.

This will involve verifying the following:1. Verify the deployed adaptor used JBI NMR standard.2. Verify the deployed adaptor provides life cycle information.3. Verify the deployed adaptor provides life cycle management interface.

4.1.2.9 [-392]: Single JVM DeploymentDiamond R2 shall provide a single JVM deployment for all Diamond JBI components to run in.

This will involve verifying the following:1. Verify all diamond components running in a single JVM instance.

4.1.2.10 [-410]: Adapter ConfigurationStatic configuration shall be provided by way of an XML-based deployment descriptor.

This will involve verifying the following:1. Verify there is an xml configuration file and its default configuration is used.2. Verify configuration changes to the deployment descriptor are not picked up.3. Verify adaptor configuration changes are picked up on restart.

4.1.3 SOAP Gateway Requirements

4.1.3.1 [-500]: 2SAP based on Apache Axis 1.2 or laterThe Diamond SOAP Gateway shall be based on the 2SAP SOAP router from Avaya Research, including Apache Axis 1.2 or later as its core Web Services processing infrastructure.

This will involve verifying the following:1. Script will verify version 1.2 or later of Axis.

4.1.3.2 [-502]: StateThe SOAP Gateway shall keep state on request and response for sending responses back to clients after features have been processed on the bus. The SOAP Gateway should maintain the message exchange patterns and asynchrony of invoked service.Notes: This session correlation function is provided by 2SAP.

This will involve verifying the following:1. Verify the SOAP Gateway keeps session information for each incoming request.2. Verify the session information is returned to the client.

4.1.3.3 [-504]: WSDL ComplianceThe SOAP Gateway shall be compliant with all mandatory features of the WSDL 1.1 specification (no proprietary extensions).

This will involve verifying the following:1. Verify that the Gateway is WSDL 1.1 compliant.2. Verify usable by other WSDL 1.1 compliant clients.

4.1.3.4 [-506]: SOAP ComplianceThe SOAP Gateway shall be compliant with all mandatory features of the SOAP 1.1 and 1.2 specifications (no proprietary extensions).

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

17

27

552553554555556557558559560561562563

564565566567568569570

571572573574575576577

578

579580581582583584585586587

588589590591592593594595596597

598599600601602603604605

606607608

282930

Page 18: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:1. Verify the SOAP Gateway is SOAP 1.1 and 1.2 compliant2. Ensure no proprietary extensions used. 3. Verify the SOAP Gateway accepts messages from clients compliant with SOAP 1.1

and 1.2.

4.1.3.5 [-508]: BPEL 1.1The SOAP Gateway shall be compliant with all mandatory features of the BPEL 1.1 specification (no proprietary extensions).

This will involve verifying the following:1. Verify the SOAP Gateway is BPEL 1.1 compliant.2. Ensure no proprietary extensions used.

4.1.3.6 [-514]: BPEL InteroperabilityThe SOAP Gateway shall be interoperable with the PXE BPEL engine using their BPEL 1.1 interfaces with no additional Avaya-specific code:Notes: I.e. shall not incorporate any WS-* interface that clients cannot process without an Avaya client.

This will involve verifying the following:1. Verify the SOAP Gateway used PXE BPEL engine using their BPEL 1.1 interface.

4.1.3.7 [-516]: Diamond Core IntegrationThe SOAP Gateway shall be integrated into the Diamond Core as a Diamond Adapter (binding component) using the Adapter Framework (JBI 1.0) that communicates with the ServiceMix NMR.

This will involve verifying the following:1. Verify log information generated when SOAP Gateway Diamond Adapter places

message on the JBI bus.2. Verify log information generated when ServiceMix NMR receives messages from the

SOAP Gateway Diamond Adapter.

4.1.3.8 [-518]: Session ManagementThe SOAP Gateway shall manage the relationship between internal Diamond sessions it creates and external sessions exposed to the invoking web service client. The SOAP gateway shall correlate communications on the Diamond bus related to this internal session to synchronous or asynchronous web services. The external session shall be managed according to the WSDL definition of the web service. Notes: For example, an external client may start a session and create an exception based conference. The SOAP gateway will create an internal SessionID for use when communicating on the JBI bus about this session. Any internal asynchronous messages sent to the SOAP gateway will contain this SessionID allowing the SOAP gateway to correlate the response to the correct external client session. (Break up into several test cases)

This will involve verifying the following:1. Verify all messages sent to and from an external web service have the same session

ID.2. Verify multiple clients have unique IDs and responses from the SOAP Gateway are

sent to the correct client.

4.1.3.9 [-520]: Remote ExecutionThe SOAP Gateway shall forward all incoming SOAP requests on to the Adapter Framework (JBI bus) for remote execution of the request by other Diamond services (e.g. the Orchestration Engine, although the exact service will be selected by the NMR), following JBI NME (Normalized Message Exchange). The SOAP Gateway shall be able to relay events from its Adapter to the clients.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

18

31

609610611612613614615

616617618619620621622623

624625626627628629630631632

633634635636637638639640641642643

644645646647648649650651652653654655656657658659660661

662663664665666667

323334

Page 19: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:1. Verify all messages that are received by the SOAP Gateway and are of the correct

format are forwarded to the JBI BUS.2. Verify the SOAP Gateway waits to receive a response from the JBI Bus concerning

the outcome of the remote request.

4.1.3.10 [-522]: SOAP Gateway ReportingThe SOAP gateway shall expose MBeans providing usage information.

This will involve verifying the following information can be retrieved from the MBean:1. Incoming Requests2. Outgoing Responses3. Time for servicing requests4. # of synch / async requests/events (recorded for a day by day evaluation of busy hour

calls)5. Active requests in progress6. Memory Usage for the threads or thread groups in use by the SOAP gateway

4.1.3.11 [-524]: Timeout Service ExecutionThe SOAP Gateway shall provide the capability for timing out the response for a service request. The specific time out period shall be configured on a per service basis and is defined in the Diamond Application SRAD.

This will involve verifying the following:1. Verify the default timeout waiting for a response to the SOAP Gateway is adhered to.2. Verify the SOAP Gateway timeout can be reconfigured by the xml configuration file. A

service/restart is required. 3. Test the boundary values for this value. These values need to be specified. 4. Test incorrect information.

4.1.3.12 [-526]: HTTPSThe SOAP Gateway shall support HTTPS transport for encrypted communication. WSDL definitions of Diamond web services shall include a conformance claim for Transport Layer Security, specifying the conformance URI of http://ws-i.org/profiles/basic-security/transport/1.0.

This will involve verifying the following:1. Verify the WSDL advertises the HTTPS conformance claim for TLS. 2. Verify the necessary security certificates are available to the SOAP Gateway.3. Verify the SOAP Gateway is listening on the HTTPS port (default).4. Using a SOAP client that provides Secure HTTP communication verify a message

can be submitted successfully through the Gateway.

4.1.3.13 [-528]: HTTP AuthenticationThe SOAP Gateway shall support HTTP Basic authentication (over HTTPS) using JAAS to authenticate the login/password against the local LDAP or enterprise LDAP for secure authentication of services. WSDL definitions of Diamond web services shall include a conformance claim for Username Token Profile, specifying the conformance URI of http://ws-i.org/profiles/basic-security/username-token/1.0.Notes: For Phase A the login/password list used for the basic authentication service should be whatever is most convenient for 2SAP development (perhaps entries in /etc/passwd). However when it’s done the passwords should not be stored in plain text.

This will involve verifying the following:1. Verify HTTP Auth is available and is support by the SOAP Gateway.2. Verify SOAP Gateway refuses connection when authentication fails.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

19

35

668669670671672673674

675676677678679680681682683684685686687

688689690691692693694695696697698699

700701702703704705706707708709710711712

713714715716717718719720721722723724725

363738

Page 20: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

3. Verify username and password stored in /etc/passwd in phase a. The password should be encrypted in the file.

4. Check that the WSDL advertises HTTP Auth capability and conformance to the above URI.

5. Verify no passwords are logged in plain text.

4.1.3.14 [-530]: WS-I Basic Security ProfileThe SOAP Gateway shall support WS-I Basic Security profile to achieve end-to-end security when necessary. WSDL definitions of Diamond web services shall include a conformance claims for the Basic Security Profile, specifying a conformance URI of http://ws-i.org/profiles/basic-security/core/1.0.

Notes: This includes authenticating the use of the web service. The Basic Security Profile is defined by http://www.ws-i.org/Profiles/BasicSecurityProfile-1.0.html.

This will involve verifying the following:1. Verifying the SOAP gateway supports WS-security.2. Verifying the exposed WSDL claims conformance with Basic Security Profile.

4.1.4 SIP Adapter Requirements

4.1.4.1 [-610]: 3261 ComplianceThe SIP Adapter shall be compliant with RFC 3261.

This will involve verifying the following:1. Verify the SIP Adapter is compliant with RFC3261

4.1.4.2 [-612]: SIP transportsThe SIP Adapter shall support the following transports:

TLS TCP UDP

This will involve verifying the following:1. Testing the SIP Adapter connecting over TLS2. Testing the SIP Adapter connecting over TCP3. Testing the SIP Adapter connecting over UDP

4.1.4.3 [-630]: SIP JAIN UsageThe SIP Adapter shall use the SIP JAIN interface.Notes: The Diamond SIP adapter is based on a SIP JAIN interface, built on the Flextronics (Hughes) SIP stack.

This will be tested during click to call testing.1. Verify the SIP Adapter uses the SIP JAIN Interface.

4.1.4.4 [-640]: SIP Standards SupportThe SIP Adapter shall support the SIP call control for conferencing mechanisms, as specified in:

IETF Draft: SIP Call Control - Conferencing for User Agents. Current version is: http://ietf.org/internet-drafts/draft-ietf-sipping-cc-conferencing-06.txt

This will involve verifying the following:1. The SIP Adaptor supports SIP Call Control as per draft.

4.1.4.5 [-642]: SIP Standards SupportThe SIP Adapter shall support SIP event notification, as specified in:

SIP Event Notification, as specified in RFC 3265.Notes: The B2BUA needs to be able to subscribe to a conference focus and receive the resultant events.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

20

39

726727728729730731

732733734735736737738739740741742743

744

745746747748749750

751752753754755756757758759760

761762763764765766767768

769770771772773774775776777

778779780781782

404142

Page 21: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:1. Verify the SIP Adapter supports SIP event Notification (RFC 3265)

4.1.4.6 [-643]: SIP Standards SupportThe SIP Adapter shall support the SIP REFER method, as specified in:

The REFER method, as specified in the IETF RFC 3515.

This will involve verifying the following:1. Verify the SIP REFER method works as specified in RFC 3515.

4.1.5 Management Services Requirements (new header)

4.1.5.1 [-1005]: JDK JMX ServerDiamond shall use the Java 1.5 JDK JMX server as its JMX provider.

This will involve verifying the following:1. Retrieve the version of the JMX Server running.2. Verify the JMX server can be contacted using jconsole.

4.1.6 Performance Requirements

4.1.6.1 [-800]: Local Performance RateThe JBI bus shall provide a local message rate (from one component to another, both in the same JVM) of no less than 1500 messages per second.Notes: This throughput will be verified via load testing and there will be no corresponding run time diagnostics or alarms.

This will involve verifying the following:1. Verify 1500 messages per seconds can be sent between components.

4.1.6.2 [-820]: ConcurrencyThe JBI bus shall support no less than 50 connected components.

This will involve verifying the following:1. Verify the JBI bus can have more than 50 connected components connected at once.

4.1.6.3 [-830]: Start-up TimeThe JBI bus shall be available for use no more than 90 seconds from boot time.Notes: This requirement specifically pertains to JBI itself, not to the overall Diamond R2 application.

This will involve verifying the following:1. Verify the SOAP Gateway and JBI bus are accepting messages within 90 seconds of

starting.

4.1.6.4 [-840]: Maximum Request TimeThe JBI bus shall never exceed more than 100 msec to process a single request. If the 100 msec threshold request is crossed, an alarm shall be raised. The 100 msec is desirable to be configurable for future change.Notes: Ideally ServiceMix exposes a performance monitoring MBean that can be used for this (periodically queried from whatever component starts up ServiceMix). The point is to have some alarm if the JBI system is poorly performing. If some other indicator can be more conveniently obtained, this requirement may be MRd.

This will involve verifying the following:1. Verify the JBI Bus does not exceed 100ms to process a single request.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

21

43

783784785786

787788789790791792793

794

795796797798799800801

802

803804805806807808809810811

812813814815816817

818819820821822823824825826

827828829830831832833834835836837444546

Page 22: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

4.1.6.5 [-850]: SOAP Gateway performanceOn an otherwise idle system, the SOAP Gateway shall consume no more than 50ms to route in an incoming web services request to the Diamond adapter.Notes: This requirement applies to the time used by the SOAP gateway exclusive of any external requests it may make.

This will involve verifying the following:1. Verify it takes less that 50ms to route an incoming request from the SOAP Gateway

to the diamond adaptor.

4.1.6.6 [-860]: SOAP Gateway concurrencyThe SOAP Gateway shall support no less than 250 active synchronous requests.

This will involve verifying the following:1. Verify more that 250 synchronous requests can active via the SOAP gateway at the

same time.

4.1.6.7 [-870]: SIP B2B UA concurrencyThe SIP B2B UA shall support no less than 250 active requests.

This will involve verifying the following:1. Verify the SIP B2BUA can support more that 250 active requests.

4.1.6.8 [-1220]: Diamond LoggingDiamond shall use “Diamond” in the LogCode when calling the Logging client API.

Notes: This is used to identify the code in the messages such that they can be viewed later. Further subcategories will be specified on an “as needed” basis by development and as defined in the ISD.

This will involve verifying the following:1. Verify all diamond logging is prepended with the word “Diamond”

4.1.7 Software Platform

4.1.7.1 [-1500]: RedHat Enterprise LinuxDiamond shall run on RedHat Enterprise Linux 4 Update 2 as the operating system.This will involve verifying the following:

1. A script will retrieve OS name and version information and verify version is correct.2. Nice to retrieve other information, available memory, etc.

4.1.7.2 [-1515]: Java Version 1.5

This will involve verifying the following:1. A script will retrieve the Java version information and verify version is correct.

4.1.7.3 [-1520]: Tomcat 5.5Diamond shall require Tomcat 5.5 for use with the SOAP Gateway.

This will involve verifying the following:1. A script will retrieve the Tomcat version information and verify version is correct.

4.1.7.4 [-1525]: ApacheDiamond shall use the Apache / mod_jk connector setup with Tomcat for providing default HTTP / HTTPS access to the server, and for certificate management.Notes: Core Services certificate management is done through Apache.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

22

47

838839840841842843844845846847848849

850851852853854855856

857858859860861862

863864865866867868869870871

872

873874875876877878879

880881882883884

885886887888889890

891892893894

484950

Page 23: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:1. A script will retrieve the Apache / mod_jk connector setup version information and

verify version is correct.4.1.8 Lifecycle Management

The Lifecycle Manager is a process that handles incoming requests either from the remote interface or from Lifecycle command line utilities. The Lifecycle manager provides the ability to manage the lifecycle of processes (Avaya or third party), web services, servlets running within a Tomcat container.The implementation of the lifecycle manager itself is in Java. The implementation can then be exposed via remote interface using DSS. In a distributed environment there would be one lifecycle manager & watchdog running on every host machine. To address failure recovery of the watchdog, there will be another instance of watchdog watching the watchdog.

4.1.8.1 [-1280]: Core Services Lifecycle ManagerDiamond shall use the Core Services Phase 3.0 Lifecycle Manager for lifecycle of the JBI container.

This will involve verifying the following:1. Verify Instance of the Core Services Lifecycle Manager is running.

4.1.8.2 [-1282]: WatchdogDiamond shall use the Core Services Phase 3.0 Watchdog for lifecycle of the JBI container.

This will involve verifying the following:1. Verify Instance of the Watchdog (or 2) is running.

4.1.9 Remote Access

4.1.9.1 [-1255]: Default Customer LoginDiamond shall provide a default customer login of “cust” as both a Linux account and an OAM user. The Diamond installer is required to prompt to set a new password for this user as part of the installation.

This will involve verifying the following:1. User “cust” exists as a Linux account and login successfully2. User “cust” exists as an OAM user and login successfully3. Verify the diamond installer prompts to set the password during install for OAM user.4. Verify the diamond installer prompts to set the password during install for Linux user.

4.1.10 Secure Access

The Diamond R2 release is a single box solution, where the security architecture is to harden that box and secure all remote interfaces, and protect against any possible attacks, intrusion, and potential service outages. There are essentially four inputs to the Diamond R2 server: the admin user, the service user, the customer application, and the SIP network. All of these can and will support secure encrypted channels for communication as shown below.

4.1.10.1 [-1300]: Secure Admin AccessDiamond shall only support HTTPS for administrative access to the server.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

23

51

895896897898899

900

901902903904905906907908909

910911912913914915916

917918919920921922

923

924925926927928929930931932933934935

936

937938939940941942943

944945

525354

Page 24: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Notes: Providing a simple redirect from the normal HTTP port to the HTTPS administrative login form is perfectly OK.

This will involve verifying the following:1. HTTP port is redirected to HTTPS port2. HTTP -> HTTPS redirect is to the HTTPS administrative form3. Can successfully login to administrative form

4.1.10.2 [-1302]: Secure Application AccessDiamond shall only support HTTPS for application access to the server.This will involve verifying the following:

1. HTTP access not available for Customer Application2. HTTP access not available for Administration Application3. HTTPS access available for Customer Application4. HTTPS access available for Administration Application

4.1.10.3 [-1304]: Secure Shell AccessDiamond shall only support SSH for shell access to the server.Notes: SFTP and SCP will be provided consistent with Core Services.This will involve verifying the following:

1. SSH access is available to the server2. SFTP access is available to the server3. SCP access is available to the server

4.1.10.4 [-1306]: Secure SIP AccessDiamond shall support TLS as the SIP transport for the SIP adapter.Notes: The Diamond B2BUA does not directly terminate any media streams, so RTP vs. SRTP is up to the endpoints that are external to Diamond.This will involve verifying the following:

1. TLS is supported by the SIP adapter2. SIP messages are being transported over TLS

4.1.10.5 [-1308]: CertificatesDiamond shall use the Core Services certificate mechanism as defined in CID 109144. The Avaya SIP CA shall be used for the SIP B2BUA. While either the Avaya CA or customer installed certificate shall be supported for the SOAP Gateway. For phase A, Phase B for customer installed certificate for the SOAP Gateway.This will involve verifying the following:

1. AVAYA SIP CA stored on Diamond machine2. AVAYA CA used by B2BUA for SIP3. AVAYA CA used by SOAP Gateway

4.1.11 Mandatory Requirements4.1.11.1 [-1350]: Denial of Service AttacksThe system shall survive denial of service attacks without loss of sanity, without rebooting or restarting, without reloading, and shall automatically recover to full service after the denial of service attack is removed. DOS attacks that shall be protected against include, at a minimum, those described in Appendix A - Attack Definition List in Security Score Card document (COMPAS ID 104496)Source: Security Scorecard – M1 Section 3.1.1.1Release: Phase ANotes:

SYN Attack: This will be covered by the Linux TCP/IP stack. Kernel will be configured to prevent SYN attacks for turnkey. For software only this is responsibility of the customer.

TCP Attack: Each core service that opens a TCP listen socket shall handle this case where an application issues multiple TCP connects but does not send any packet over the socket connection to that core service

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

24

55

946947948949950951952953

954955956957958959960961

962963964965966967968969

970971972973974975976977

978979980981982983984985986987

988989990991992993994995996997998999

1000100110021003

565758

Page 25: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Smurf & Fraggle: This will be covered by the Linux TCP/IP stack. Teardrop, Overlap or Fragmented Packets: This will be covered by the Linux

TCP/IP stack. Ping Flood: The security script will configure PAM to prevent against ping floods. Finger of Death: finger will be disabled for the Turnkey solution Chargen Packet Storm: Port 19 will be disabled for Turnkey solution Malformed Packets or Oversized Packets: Each CS service that opens a listen

socket shall handle Malformed Packets or Oversized packets. Need to check for 0 and 60 K packet size

Rationale: This will involve verifying the following:

1. Linux Kernel is configured to prevent SYN attacks for Turnkey 2. System survives Smurf attack3. System survives Fraggle attack4. System survives Teardrop attack5. System survives Overlap attack6. System survives Fragmented Packets attack7. PAM is configured to prevent against ping floods8. System survives Ping Flood attack9. Finger is disabled10. System survives Finger of Death attack11. Port 19 is disabled12. System survives Chargen Packet Storm13. System handles malformed packets14. System handles oversized packets

Note: each of the above will only pass if the following criteria is met:a. without loss of sanity, b. without rebooting or restarting, c. without reloading, d. and shall automatically recover to full service after the denial of service attack

is removed.

4.1.11.2 [-1352]: Only software that is neededOnly those network services, applications, packages, libraries, files, operating system services, and configurations required by the system design shall be installed and enabled during system installationNotes: This requirement shall be met by the common installer. This will involve verifying the following:

1. Network Services required by system design are installed2. Applications required by system design are installed3. Packages required by system design are installed4. Libraries required by system design are installed5. Files required by system design are installed6. Operating System Services required by system design are installed7. Configurations required by system design are installed8. Network Services not required by system design are not installed or disabled9. Applications not required by system design are not installed or disabled10. Packages not required by system design are not installed or disabled11. Libraries not required by system design are not installed or disabled12. Files not required by system design are not installed or disabled13. Operating System Services not required by system design are not installed or

disabled14. Configurations not required by system design are not installed or disabled

4.1.11.3 [-1354]: No passwords transferred in the clearNo network service shall be used that transfers login password information in the clear (e.g. no unprotected Telnet, no unprotected FTP, no unprotected SNMP). Services that transfer

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

25

59

100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036

10371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059

106010611062

606162

Page 26: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

password information in the clear may be used only when protected, for example, by running the service through encrypted channelsNotes:

1. Telnet & FTP shall be disabled. 2. SSH and SFTP shall be used.3. HTTPS shall be used for request to the Apache web server and tomcat container

This will involve verifying the following:1. Telnet is disabled2. FTP is disabled3. RSH is disabled4. SSH available5. SFTP available6. HTTPS available to Apache web server and Tomcat container

4.1.11.4 [-1356]: Password for every loginThere shall be a password for every login and the capability to assign a password to every login. No login shall have a default password that is active after initial system installation and configuration -- only documented logins. There shall be no undocumented logins. (a.k.a., backdoors).Release: Phase ANotes:

- For turnkey craft and sroot will have default password created via the post install script

This will involve verifying the following:1. No undocumented logins2. Capability to assign a password to every login3. There is a password for every login4. No login has a default password after initial installation

4.1.11.5 [-1358]: No direct root accessThere shall be no direct logins to accounts with the highest privilege level (e.g. root account). Users shall have to escalate to logins with the highest privilege level as additional privileges are needed. An exception is that direct login to the account with the highest privilege is allowed from the console terminal.

This will involve verifying the following:1. No direct logins to root2. No direct logins to accounts with highest privileges3. Direct login to account with highest privilege is allowed from console terminal

4.1.11.6 [-1360]: Secure transmissionData transmission between Avaya equipment (e.g., as used by Services) and customer equipment in support of servicing said equipment must be done securely when a non-secure data network (e.g., Internet) is used. Mechanisms that ensure that the data is protected from being viewed or modified shall be used.Notes: Data transmission will only occur through secure mechanism like ssh and sftp

This will involve verifying the following:1. Data transmission between Avaya equipment and customer equipment done securely

through SSH2. Data transmission between Avaya equipment and customer equipment done securely

through SFTP

4.1.11.7 [-1362]: Administrable security secretsWhen SNMP is used, the security secrets (e.g. community strings) shall be administrable by the system administrator

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

26

63

106310641065106610671068106910701071107210731074107510761077

10781079108010811082108310841085108610871088108910901091109210931094

10951096109710981099110011011102110311041105

1106110711081109111011111112111311141115111611171118

111911201121

646566

Page 27: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:1. If SNMP is used, security strings are administrable by the system administrator

4.1.11.8 [-1364]: Documented servicesAll network services that are enabled for permanent or temporary use will be documented. Either the Usage Notes for applicable core services or the Diamond security document shall describe the need for the network service and the authentication mechanisms that must be satisfied in order to enable/disable, configure and use the network service. The intent of the documentation is to aid customers when analyzing and securing their networks.

This will involve verifying the following:1. Services are documented2. The usage notes will describe authentication mechanisms that must be satisfied to

enable/disable/configure and use the network service3. Documented services matches services on server

4.1.11.9 [-1366]: Vulnerability analysisVulnerability analysis tools shall be used to test a product/platform for resilience before the product/platform is made GA. The results of the vulnerability analysis shall be documented.Notes: This tool will be provided by the ECG Security teamThis will involve verifying the following:

1. Tool is configured and used against each server in application2. Documentation/report made for each server in application

Note: ECG tool is required before this test case can be started

4.2 Diamond Application Testing4.2.1 Click-To-Call Web Service (Diamond SRAD)

4.2.1.1 [113358-10] Click to Call Web Service Diamond R2 shall provide a Click to Call web service for initiating a call from an originator to one or more called parties. The originator and called parties shall be identified by URIs. The web service call shall be synchronous from the invoking client’s view, returning success if the originator is successfully added to the call.

This will involve verifying the following:

1. Initiate Call from originator to one party2. Initiate call from originator to more than one party3. The originator is identified by URI4. The called parties are identified by URI5. Web service call is synchronous from invoking clients view6. Originator is successfully added to the call.7. Call will fail if a non SIP URI is entered8. Call will fail if an unknown URI is entered

4.2.1.2 [113358-20] Click to Call Web Service - Multi-PartyIn the multi-party case, the Click to Call web service call shall succeed only if the conference bridge has sufficient ports available to handle the call. In addition, conferees shall be added to the conference only after they answer and only if they pass a “voicemail filter (see CID 114190)”, i.e. no call progress tones or answering machine messages shall be audible to the originator or anyone added to the conference.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

27

67

1122112311241125

11261127112811291130113111321133113411351136113711381139

11401141114211431144114511461147114811491150

11511152

115311541155115611571158115911601161116211631164116511661167116811691170

1171117211731174117511761177

686970

Page 28: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:

1. Call succeeds if bridge has sufficient ports2. Call fails if bridge has insufficient ports3. Conferee is added to call after they answer and pass voicemail filter(no voicemail,

real answer)4. Conferee not added to call if fails voicemail filter5. No call progress tones audible to the originator6. No answering machine messages audible to the originator7. No call progress tones audible to other parties in the conference8. No answering machine messages audible to other parties in the conference9. For a three party call, if an unknown URI for one party is entered the conference will

continue10. For a three party call, if two unknown URI’s are entered then the conference call fails11. For a three party call, if one party doesn’t answer then the call will continue with two

parties12. For a three party call, if two parties don’t answer then the call fails13. If the originators SIP URI is incorrect the call will fail.14. Failures are logged

4.2.1.3 [113358-30] Click to Call AuthenticationThe Click to Call web service shall accept requests from an administered list of clients (ACL) and only if the client successfully authenticates with an HTTP authentication mechanism.

This will involve verifying the following:

1. Click to Call web service authenticates client with HTTP authentication mechanism2. Click to Call web service accepts requests from administered list of clients3. Click to Call web service rejects requests from client not on administered list of clients

4.2.1.4 [113358-40] Click to Call BillingAll calls made by an invocation of the Click to Call web service shall be billable to the originator identified in the web service request.

This will involve verifying the following:

1. Check billing info on Crystal is from originator2. Billable item is identified to the originator of the web service request.3. No Billable items are identified to the receiver of the web service request

4.2.2 B2BUA Interface4.2.2.1 [113358-15] Click to Dial StandardDiamond R2 shall follow the click to dial standard as defined in section 2.19 of the SIPPING Service Examples document (draft-ietf-sipping-service examples-09). This follows RFC 3515 (REFER).Nb: the current version of this document is draft-ietf-sipping-service examples-10

This will involve verifying the following:

1. Each B2BUA SIP message follows the format set out in the Click to Dial section of the current SIPPING Service Examples document.

2. The flow used by the B2BUA for a Click to Dial will match the flow specified in the Click to Dial section of the current Sipping Service Examples document

4.2.2.2 [113358-16] Invite to Conference FactoryDiamond R2 shall be capable of performing an INVITE to the conference factory URIs, and retrieving the actual conference location from the 302 response.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

28

71

117811791180118111821183118411851186118711881189119011911192119311941195119611971198

1199120012011202120312041205120612071208

1209121012111212121312141215121612171218

12191220122112221223122412251226122712281229123012311232

123312341235

727374

Page 29: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:

1. Diamond R2 can perform an INVITE to the conference factory URIs2. Diamond R2 can retrieve the conference location from the 302 response.

4.2.2.3 [113358-17] Out of Dialog REFERDiamond R2 shall be capable of performing an out of dialog REFER to the conference bridge for performing outdials on Crystal.

This will involve verifying the following:

1. Diamond R2 can perform an out of dialog REFER2. An out dial is performed on the Crystal Bridge. We can verify outdial happened by

checking the Crystal bridge (CLI).

4.2.3 Deployment Models4.2.3.1 [113358-2430] SES/CM InteroperabilityThe Diamond Application shall interoperate at a SIP level with Avaya’s SIP solution (SES and CM). As a click to dial or click to conference originator, it is possible to be a SIP phone or non SIP phone (behind CM) through the SES proxy. A click-to-call or conference participant likewise can be a SIP phone on SES or a CM phone via the CM SIP trunk. Testing shall be performed for both Home and Edge Proxy configurations.

This will involve verifying the following:

1. Diamond interoperates with SES/CM at a SIP level (Home Proxy configuration)2. Diamond interoperates with SES/CM for a click to dial (Home Proxy configuration)3. Diamond interoperates with SES/CM for a click to conference (Home Proxy

configuration)4. A Click to Call participant can be a SIP phone on SES (Home Proxy configuration)5. A Click to Call participant can be a CM phone via the CM SIP trunk (Home Proxy

configuration)6. Diamond interoperates with SES/CM at a SIP level (Edge Proxy configuration)7. Diamond interoperates with SES/CM for a click to dial (Edge Proxy configuration)8. Diamond interoperates with SES/CM for a click to conference (Edge Proxy

configuration)9. A Click to Call participant can be a SIP phone on SES (Edge Proxy configuration)10. A Click to Call participant can be a CM phone via the CM SIP trunk (Edge Proxy

configuration)

4.2.4 Installation and Upgrades4.2.4.1 [113358-1700] Platform RequirementsThe Diamond R2 Phase A and B solution will run on systems that meets theFollowing minimum requirements:

IBM x306m box or equivalent Redhat Enterprise Linux 4.0 Update 2 for Phase A SMP Kernel 2.6 Hyperthreading enabled. Memory 1 GB

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

29

75

1236123712381239124012411242

12431244124512461247124812491250125112521253

1254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281

1282128312841285128612871288128912901291

767778

Page 30: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:

1. Server is IBM Server x306m2. Operating system is Redhat Enterprise Linux 4.0 Update 23. SMP Kernel version is 2.64. Hyperthreading is enabled5. Memory is 1GB

4.2.4.2 [113358-1710] Diamond Application CDThe Diamond R2 Phase A and B solution shall include a Diamond Application CD that is installed on to a system that has already undergone the Operating System CD installation The Diamond 3956 Application CD shall contain the following software:

Diamond Core [service bus] Diamond Adapters [B2BUA] VIA Email User Management BPEL CoreServices OA & M and dependencies Logging Alarming (Phase B) Backup / Restore (Phase B) Watchdog / Lifecycle Manager WebLM (Phase B) Diamond Admin Application (Phase B) Diamond Application Installation Script (ICF) Diamond Security Module Required Third Party Packages [e.g. Tomcat, Java etc]

This will involve verifying the following:

1. Diamond Core on installation CD2. Diamond Adapters on installation CD3. VIA on installation CD4. Email on installation CD5. User Management on installation CD6. BPEL on installation CD7. Core Services OA & M and dependencies on installation CD8. Logging on installation CD9. Watchdog/Lifecycle Manager on installation CD10. Diamond Application Installation Script (ICF) on installation CD11. Diamond Security Module on installation CD

4.2.4.3 [113358-1715] Diamond 3rd party packageDiamond shall use the following 3rd party packages (some of from Cores Services and some will be supplied by Diamond).From Core Services (version available is CS 3.0)

J2SE 1.5.2_02 Tomcat 5.5.9 Axis 1.2 Mod_jk 1.2.6 Apache 2.2.13-4 Mon Tripwire 2.3.1-20.fdr.1 OpenLDAP 2.2.13-4 (this includes openldap, openldap-servers and

openldap-clients RPMs) Postgres 8.1

Diamond provided: ServiceMix JBI Bus

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

30

79

12921293129412951296129712981299

130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335

1336133713381339134013411342134313441345134613471348134913501351

808182

Page 31: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

PXE BPEL Engine

This will involve verifying the following:

1. Core Services contains J2SE 1.5.2_02 on CD2. Core Services contains Tomcat 5.5.9 on CD3. Core Services contains Axis 1.2 on CD4. Core Services contains Mod_jk 1.2.6 on CD5. Core Services contains Apache 2.2.13-4 on CD6. Core Services contains Mon on CD7. Core Services contains Tripwire 2.3.1-20.fdr.1 on CD8. Core Services contains OpenLDAP 2.2.13-4 (this includes openldap, openldap-

servers and openldap-clients RPMs) on CD9. Core Services contains Postgres 8.1 on CD

Diamond provided:10. Diamond contains ServiceMix JBI Bus on CD11. Diamond contains PXE BPEL Engine on CD

Note: may have to check post install

4.2.4.4 [113358-1720] ICFThe Diamond R2 Phase A and B solution shall use the Installation Configuration Framework [ICF] from Core Services Phase 3.0 for performing its installation and upgrades.

This will involve verifying the following:

1. ICF installer is being used to install Diamond2. Installer conforms to Avaya’s Common Look and Feel standards3. Diamond is installed4. Rollback is supported5. Upgrade is supported6. Uninstall is supported7. Patches are supported

4.2.4.5 [113358-1730] ICF Linux OnlyThe Diamond R2 Phase A and B solution shall be deployed on Linux only. The ICF configuration shall verify the OS is RHEL 4 and has SMP Kernel 2.6.

This will involve verifying the following:

1. Verify Diamond is installed on system with RHEL 4 and has SMP Kernel 2.62. Verify Diamond won’t install on non conforming OS3. Try installing on a Windows machine

4.2.4.6 [113358-1740] ICF Console OnlyThe Diamond R2 Phase A and B solution shall only use the console mode for the ICF based application installation.

This will involve verifying the following:

1. Diamond shall be installed using ICF in console only mode

4.2.4.7 [113358-1830] Backup and Restore prior to upgradesData should be backed up manually before upgrades and restored after upgrade is complete.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

31

83

135213531354135513561357135813591360136113621363136413651366136713681369137013711372

1373137413751376137713781379138013811382138313841385138613871388

13891390139113921393139413951396139713981399

140014011402140314041405140614071408

14091410

848586

Page 32: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

This will involve verifying the following:

1. Data can be backed up before an upgrade2. Data can be restored after an upgrade3. Data restored successfully

4.2.4.8 [113358-1850] ICF will provide rollback capabilities for Diamond product after an upgrade.

The Diamond R2 Phase A and Phase B installer should provide capabilities to rollback to the previous version of the product if desired. The installer will not provide rollback capabilities for individual modules within the product.

This will involve verifying the following:

1. ICF provides feature to rollback to previous version2. ICF rolls back to previous version successfully3. There are no features to rollback individual modules within the product.4. Data is preserved across rollback5. Schema is preserved across rollback

4.2.4.9 [113358-1860] Data restore after rollback.Data backed up using backup and restore utility will be used to restore it after rollback..

This will involve verifying the following:

1. Backup is successful2. Restore is successful

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

32

87

141114121413141414151416141714181419

1420142114221423142414251426142714281429143014311432143314341435

1436143714381439144014411442144314441445

888990

Page 33: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

5 FEATURES NOT TO BE TESTED BY SVAvaya SV will not be verifying the following Features:

5.1 Crystal

Crystal verification is outside the scope of Diamond R2: Phase A

5.2 SES

SES verification is outside the scope of Diamond R2: Phase A.

5.3 CM

CM verification is outside the scope of Diamond R2: Phase A.

5.4 Diamond Core Testing

List of requirements outside the scope or being tested elsewhere of Diamond R2: Phase A.

5.4.1 [-501]: Standard Axis Deployment

Reasons for not testing (supporting documentation):Questions on SRAD from Test Plan Review spreadsheet:Note made by Frederick Block: Wu’s team should test this

5.4.2 [-400]: Vendor Independent JBI Integration

Reasons for not testing (supporting documentation):Questions on SRAD from Test Plan Review spreadsheet:

Note made by Frederick Block: I don't think there's a way to test this with "black box" testing. This is really a design constraint allowing use of only "pure JBI" interfaces that should be enforced at code inspection time.

5.5 Diamond Application Testing

List of requirements outside the scope or being tested elsewhere of Diamond R2: Phase A.

5.5.1 [113358-2160] Login Session Properties

Reasons for not testing (supporting documentation):

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

33

91

1446

144714481449

1450

145114521453

1454

14551456

1457

14581459

1460

1461

1462

1463146414651466

1467

14681469

1471147214731474

1475

1476

1477

14781479

929394

Page 34: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Questions on SRAD from Test Plan Review spreadsheet:Note made by Frederick Block: I think all the OA&M requirements are phase B.

5.5.2 [113358-1840] Databases (postgres and LDAP) should be preserved across upgrades.

Reasons for not testing (supporting documentation):Questions no SRAD from Test Plan Review spreadsheet:Note made by Frederick Block: Phase B.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

34

95

14801481

14821483

1484148514861487

969798

Page 35: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

6 TEST STRATEGY / APPROACHThe following section details the test strategy and process to be followed by SV. There will be three Stages to testing as described below:

Stage 1 – Setup Stage 2 – 100% Coverage Testing Stage 3 – Regression Testing

SV will execute 100% Coverage testing against a diamond

6.1 Stage 1 (Setup)

Installation of Servers

Diamond RHEL 4.0 Diamond software from CD

SES RHEL 8.0 ( on x306m it’s 4.0 ) SES Software installation

Voice Portal RHEL 3.0 Installation of VP software

CM Configuration of CM

6.2 Stage 2 (100% Coverage Test Cycle)

This release would ideally be a single drop with all features (as per Section 3) implemented. Due to time to market pressures this will not be possible and will thus consist of three drops:

Drop 1 – will consist of iteration 7 and before requirements plus those implemented as part of Iteration 8

Drop 2 – will consist of drop 2 requirements plus those implemented as part of

Iteration 9

6.3 Stage 3 (Regression Test Cycle)

The Regression Test Cycle will consist of a selection of test cases that will be run against the Final GA build. Test cases to be run will be determined using the following criteria:

Numbers of bugs found in a particular component Level of complexity of code in a particular component Amount of code churn in a particular component

If for some reason this build fails the Regression Test Cycle, the product will be required to undergo another Regression Test Cycle.

6.4 Entrance Criteria

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

35

99

1488148914901491149214931494149514961497

1498

1499150015011502150315041505150615071508150915101511151215131514

1515

15161517151815191520152115221523

1524

1525152615271528152915301531153215331534

1535

100101102

Page 36: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

The following criteria need to be met before entering each Test Cycle:

6.4.1 Stage 1 (Setup)

As the main purpose of this release is to get the SV lab up and running, an installable version of the system with some dial functionality is required

6.4.2 Stage 2 (100% Coverage) Iterations 1-9 are code complete, & feature complete (as defined in Wiki). All components have complete Unit / Integration test reports All components are integrated on the main code branch

6.4.3 Stage 3 (Regression) SV complete 100% coverage as defined in section 5.2 All Severity 1 bugs are fixed All Severity 2 bugs are fixed except those that have got a wavier All Severity 3 bugs are fixed except those that have got a wavier Agreed metrics have been met by the product

6.5 Exit Criteria

The following criteria need to be meet before exiting each Test Cycle:

6.5.1 Stage 2 (100% Coverage) SV complete 100% coverage All Severity 1 bugs are fixed All Severity 2 bugs are fixed except those that have got a wavier All Severity 3 bugs are fixed except those that have got a wavier The Product meets agreed metrics as defined in Section 6.2.

6.5.2 Stage 3 (Regression) SV complete full coverage All Severity 1 bugs are verified/closed All Severity 2 bugs are verified/closed except those that have got a wavier All Severity 3 bugs are verified/closed except those that have got a wavier The Product meets agreed metrics as defined in Section 6.2.

NOTE: Severity definitions are defined in “GCS Problem Report Severity Definitions” (Compas ID 109225)

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

36

103

1536

1537

153815391540

15411542154315441545

1546154715481549155015511552

1553

15541555

1556155715581559156015611562

156315641565156615671568156915701571

104105106

Page 37: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

7 PASS / FAIL CRITERIAPlease see “GCS Problem Report Severity Definitions” (Compas ID 109225) for the problem report severity definitions that will be used when verifying this product.

7.1 Bug Analysis / Reporting

Test results will be available on a Web Page (http://sqaconference.spectel.com/) and each failed test case will have a corresponding MR. Test Coverage (i.e. the percentage of Total test cases executed) will be described by the following colour codes:

Green = 70% - 100% CoverageYellow = 50% - 69% CoverageRed = 0% - 49% Coverage

7.2 Quality Metrics

The Product is required to meet the following metrics: Item Goal

Alpha OK-to-Qualify (Beta)

Stage 2Exit Criteria

Stage 3 Exit Criteria

OK-to-Launch

Open Critical (Severity 1) Defects

TBD 0 0 0 0

Open Total (all) Defects

TBD <75 <50 <35 <35

Minimum % SV tests executed

60% 80% 100% 100% (of Regression Tests)

100% Coverage and all Regression Complete

Minimum % SV tests passed of those executed

75% 80% 90% 90% 90%

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

37

107

1572157315741575

1576

15771578157915801581158215831584

1585

15861587

1588

108109110

Page 38: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

8 SUSPENSION CRITERIA AND RESUMPTION REQUIREMENTS

This criterion is applied when one or more of the project requirements do not pass one of the exit criterions. (See Section 6.0 Item Pass/Fail Criteria)

8.1 Suspension Criteria

Testing is temporarily suspended when one or more of the following conditions occur:

Avaya Release Build not able to be installed Instability of the Avaya Release Build Hardware problem resulting in the system being down Hardware is unavailable for test Entrance Criteria not being met

8.2 Resumption Requirements

Testing will resume in full, including all subsequent tests not yet started, immediately after removal of the conditions causing suspension. This assumes that there is a need for test continuance. Including, tests not yet completed for the remaining test items. In the event of suspension due to environmental issue, testing will resume in full as soon as the environmental issue has been resolved.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

38

111

1589

1590159115921593

1594

15951596159715981599160016011602

1603

160416051606160716081609

112113114

Page 39: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

9 TEST DELIVERABLEThe following are test deliverables;

The Test Plan (this document) Test Case Procedures/Specifications (Whichever is applicable) Weekly Test Summary reports, Test logs & bug logging

9.1 Test Design Specifications

If any test tools, test automation or test suites are being developed, Test must provide a detailed Test Design Specification.

ATF Diamond R2 Phase A design specification

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

39

115

1610161116121613161416151616

1617

1618161916201621

116117118

Page 40: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

10 TESTING TASKS / MILESTONES

10.1 Testing Tasks

Testing Task Responsibilities and Scheduled Completion DateTask/Area Responsibility Scheduled

Start DateScheduled Completion

DateTest Documentation Name

Test Plan Mark Sheridan, Aoife O’Halloran, Sheamus O’Halloran

08/05/06

Test Specification SV Diamond 22/06/06

Test Coverage NameStage 1 (Set-up) SV DiamondStage 2 (100% Coverage) SV Diamond

Iterations 1-8 SV DiamondIteration 9 (plus Iteration 1-8 Regression)

SV Diamond

Stage 3 (Regression) SV Diamond

Test Report NameTest Report Doc including all Test Results

10.2 MilestonesDate Milestone Comments

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

40

119

1622

1623

1624

1625

1626

1627

120121122

Page 41: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

11ENVIRONMENTAL NEEDS11.1 Hardware

The following hardware is required to complete testing within a timely manner:

Crystalo 4 IBM x336 Servers

Diamondo 7 IBM x306m Servers

Voice Portalo 4 IBM x306m Servers

SESo 4 IBM x306 Servers

CMo 4 CM Media Switcheso 4 Media Servers

11.2 Software

The following software is required to start/complete testingList of software

Note: All software require sufficient licenses

11.3 Other Dependencies

The test effort requires lab support for servers and internal/external network access. Other dependencies include:

Timely access to default Bridge configurations Install guides are available for applications

12 RESPONSIBILITIESSee Section 13.0 Staffing & Training.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

41

123

1628

1629

163016311632

163316341635

163616371638

163916401641

164216431644

164516461647

1648

16491650165116521653

1654

1655165616571658

165916601661

124125126

Page 42: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

13 STAFFING AND TRAINING

13.1 Staffing

Name Role/Area % Commitment DurationAzgar Baig SQA Engineer 100% 6 weeksSheamus O’Halloran SQA Engineer 100% 6 weeksMark Sheridan SQA Engineer 100% 6 weeksMichelle Slattery SQA Engineer 100% 6 weeksStefano Vozza SQA Engineer 100% 6 weeksYing Zheng SQA Engineer 100% 6 weeksAoife O’Halloran SQA Mgr Engineer 75% 6 weeks

13.2 Training

Description Required For? Training ResourceCM SV Diamond Mahesh NarasimhanBPEL SV Diamond Ganesh

AnanthakrishnanServiceMix SV Diamond John SloanJDK 1.5 SV Diamond Nicholas Dronen

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

42

127

1662

1663

1664

1665

1666

16671668

128129130

Page 43: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

14 SCHEDULE

14.1 Schedule

These are approximate timings; better estimates will be available once the Test Cases have been completed.

14.1.1 Stage 1 - Setup

Test Item/Area Total Execution

Time (hr/days)

Comments

Lab SetupInstallation 2 Some parts of the

installation may require help from the developers

Test scripts for verifying install versions (Core testing)

3

Approximate Test Cycle Time: 5 man days

14.1.2 Stage 2 – 100% Coverage

Test Item/Area Total Execution

Time (hr/days)

Comments

Development to ATF 4Diamond Application creation of ATF scripts

4

Core security R&D and configuration includes ATF automation

4

Application SRAD testing including configuration of ATF automation

4

Core Requirements 3Adapter Framework Requirements

4

SOAP Gateway Requirements

4

SIP Adapter Requirements 4Management Services Requirements

4

Performance Requirements 4Software Platform Requirements

4

Lifecycle Management 4Remote Access 4Secure Access 4

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

43

131

1669

1670

1671

1672167316741675

1676

1677

167816791680

1681

132133134

Page 44: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

Mandatory Requirements 4Click to Call Web Service 4Deployment Models 4Installation and Upgrades 4Diamond R2 Operation, Administration and Maintenance

4

Approximate Test Cycle Time: 76 man days

14.1.3 Stage 3 – Regression

Test Item/Area Total Execution

Time (hr/days)

Comments

Diamond Application execution of test scripts

5

Core Regression testing 5

Approximate Test Cycle Time: 10 man days

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

44

135

16821683

1684

1685

16861687168816891690169116921693

136137138

Page 45: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

15 RISKS AND CONTINGENCIESThis section of the plan addresses the current risks and contingencies for this Master test plan.

15.1 Risks

1. Lab resources: Test Schedule could be impacted if all the hardware is not in place at the start of testing.  

2. Tools/technology and test development: May be impacted if it is necessary to cover additional features not identified in the test plan.

3. Software licensing dependencies: Many applications and tools require licenses. Insufficient number of licenses, or the expiration of existing licenses can impact the development and execution of test suites.

4. Other/third party test suites/applications: Support by the vendor may be necessary

5. Product configuration: Development had some difficulty setting up the SES server.

15.2 Contingencies

The contingencies associated with each risk are:

1. Lab resources:  Leverage resources from other groups whenever possible.   2. Tools/technology and test development: Add additional resources and/or

extend schedules. 3. Software licensing dependencies: Acquire additional licenses or renewal of

existing software licenses.   4. Other/third party test suites/applications: Acquire a support contract from

the vendor. 5. Product Configuration: Ask expert from Development to support SQA during

the configuration phase.

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

45

139

1694169516961697

1698

16991700170117021703170417051706170717081709

1710

1711

1712

171417151716171717181719172017211722

1723

1724

140141142

Page 46: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

16 REVIEWS AND APPROVALS16.1 Document Approval Sign Off

Name Signature DateMahesh Narasimhan

Pauric CreggFrederick BlockBrendan Flood

Dennis KornbluhJoann Ordille

Praveen MamnaniWu ChouArt Young

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

46

143

1725

1726

172717281729

144145146

Page 47: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

17 REVISION HISTORY 

Version Modifications Author Date1.0 Initial M.Sheridan/A.O’Halloran/S.O’Halloran 8th June /06

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

47

147

1730

1731

17321733

1734

148149150

Page 48: 2offutt/rsrch/avaya-diamond/diamon…  · Web viewVerify all diamond logging is prepended with the word “Diamond” Software Platform [113240-1500]: RedHat Enterprise Linux. Diamond

19/05/2023 SV Test Plan for Diamond R2 Application Phase A Development

18 APPENDICES

18.1 Appendix 1

Avaya Proprietary

Editor: M. Sheridan, A. O’Halloran, S. O’Halloran

48

151

1735

1736

173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772

152153154


Recommended