Smart Cloud Monitoring 7.2 Test Coverage Document · Capacity Planner – IBM Required Summary of...

Tivoli Software SmartCloud Monitoring 7.2 Overall Test Plan

Online Version is the Master Page 1� of 14

SWG - TIVOLI

Smart Cloud Monitoring 7.2 Test Coverage Document

Document Version: 0.5

Document Revision Status: Draft

Notice: The online version is the Master. Any printed versions of this document are FOR REFERENCE ONLY. Users of this document are responsible for using the official version, and for verifying that any copies are complete and of the official version.



Document Control Information

Document Owner: Document Author: Amit Tripathy Document Approvers:

Functional Area and Role Approver’s Name

SVT Test Manager

Release Manager

Document Reviewers:

Functional Area and Role

Status Reviewer’s Name

Project Manager - IBM Required Project Manager - PSL Required SVT Test Lead - IBM Required

Dev Lead - PSL Required

Dev Lead - PSL Required

Capacity Planner – IBM Required

Summary of Changes: The Document Author/Owner is authorized to make the following types of changes to the document without requiring that the document be re-approved:

• editorial, formatting and spelling

• clarification

• document structure

To request a change to this document, contact the Document Author or/ Owner. Changes to this document are summarized in the following table in chronological order.

Version Date Short Description

0.5 09/26/2012 Initial Draft

0.6 10/15/2012 Revision 1

Document Source: The official version of this controlled document is stored in the RTC Epic- 28878 Document Template: TPDM Test Plan Template TIV-T&F-0090 revision 1, Tivoli Document Library (QMX) All controlled documents must be controlled following the SWG Document Control Procedure, Proc-0006, located in the SWG Document Library (QMX) database.



About this Document The Test Coverage document describes the planned activities to ensure through a formal test activity that the new and changed functions, components, scenarios, etc. of a product/component meet the functional and nonfunctional specifications as well as customer’s / consumer’s expectations as defined in the software design. The main goal of this document is to share it with IBM Smart Cloud Monitoring Customers in order to validate test scenarios and test environments. The focus of this document is on Scenarios Verification Test (SVT) which is a formal testing activity that ensures that the software product under test is ready for customer consumption. This is accomplished by assessing how well the product will meet the persona's goals and scenarios. While the Component Verification Test (CVT) activities verify the goals of individual components, the SVT activities validate defined system designs and scenarios. CVT scenarios are not included in this document. SVT focuses on the software’s behavior in environments reflecting real or anticipated customer environments. The SVT activity verifies many aspects of the customer total experience, such as: installation, migration, performance, scalability, error recovery and others which are part of the customer configuration and operating environment. The Test Plan is formally reviewed by relevant parties as planned. This Plan is approved by the appropriate authority and published on SMC for general viewing.



Table of Contents

1.0 Introduction ...............................................................................................................................................................................5

2.0 Test Scenarios ............................................................................................................................................................................6

2.1 System Verification Test (SVT) ...............................................................................................................................................6 2.1.1 SVT Scenario 1 – VMware Agent Verify Self Describing Agent support.........................................................................6 2.1.2 SVT Scenario 2 – Verify VMware agent new attribute group Cluster_DRS_Faults .........................................................6 2.1.3 SVT Scenario 3 – HBA Performance from 2 new Storage metrics. ..................................................................................7 2.1.4 SVT Scenario 4 – Verify HBA Capacity for Host Fiber Channel and Host Internet SCSI................................................7 2.1.5 SVT Scenario 5 – Verify new attribute group “VM_Disk_Performance”. ........................................................................7 2.1.6 SVT Scenario 6 – Verify VMware agent new attribute group “Virtual_Machine” & “Datastore Cluster” .......................8 2.1.7 SVT Scenario 7 – Verify VMware agent new attribute group “percent free space”..........................................................8 2.1.8 SVT Scenario 8 – Verify VMware agent modifications to attribute group “Datastore” ....................................................8 2.1.9 SVT Scenario 9 – Capture virtual center tasks that have modified a managed entity........................................................8 2.1.10 SVT Scenario 10 – VMware Agent Affinity Test .........................................................................................................9 2.1.11 SVT Scenario 11 – VMware Agent Coexistence with other agents after upgrade........................................................9 2.1.12 SVT Scenario 12 – Verify VMware Agent Prereq checker results ...............................................................................9 2.1.13 SVT Scenario 13 – VMware Agent function after ITM infrastructure upgrade............................................................9 2.1.14 SVT Scenario 14 – VMware Agent verification with agents and Infrastructure on one server ..................................10 2.1.15 SVT Scenario 15 – VMware Dashboard and Capacity Planner virtualization data verification .................................10 2.1.16 SVT Scenario 16 – Verify VMware Agent Error Recovery........................................................................................10 2.1.17 SVT Scenario 17 – VMware Dashboard Error Recovery Testing...............................................................................11 2.1.18 SVT Scenario 18 – Agent Reports Verification ..........................................................................................................11

3.0 Platform Coverage...................................................................................................................................................................12

3.1 ITM coverage........................................................................................................................................................................12

3.2 TEMA+RTEMS Coverage ....................................................................................................................................................13

3.3 TIP/TCR Coverage ...............................................................................................................................................................13

3.4 Capacity Planner Databases Coverage................................................................................................................................13

3.5 Web Browser Coverage ........................................................................................................................................................14



1.0 Introduction The main objective of IBM Smart Cloud Monitoring (referred also as SCM from now on) is to provide a complete solution for monitoring the health of clouds and virtual environments. By “health” we mean the availability, performance and capacity of the resources of the virtual infrastructure of clouds and data centers as well as of the workloads and cloud services running in the clouds. SCM is based on an integrated set of strategic IBM/Tivoli technologies and products. The figure below provides a view of the high level architecture of SCM today. SCM has a 3-tier architecture (from top):

� Web 2.0 dashboards, reports and analytics (e.g. capacity planning) (TIP based) � SCM server (ITM v6 based) � SCM agents (ITM for Virtual Environments and ITM OS agents)

Director

TPC

… Servers, storage, network

IBM (hybrid and non-hybrid) and non-IBM Platforms (distributed and z)

TADDM

SCM Dashboards

IT System Admin

Capacity PlannerOperations

Displays TADDM

views in context

(config changes)

Platform mgmt tools (Virtual Center, IBM Systems Director, DFM, etc)

User Interface (DoJo, etc)

TCR / TIP

(WAS)

DashboardsCognos

TDW

TPC agent

ITM/ITMfVE

(monitoring server)

NMA agentNetAppagent

Hyper-V

agent

Reports

Capacity PlanningData provider

Configuration, metrics

Data Mart

Events

SCM Reports

SCM Client ( Browser)

ITM OS

agents

TEP

Citrix

Agents

KVM

Agent

VMware

agent

ITM/ITMfVE Agents

SCM

LiC (drill-down)

LiC(drill-down)

sensors

ITPA (predictive)

Forward

dynamic

changes

Import DLAs

Cisco UCS Agent

OMNIbusForward

Events (EIF)

Federated

SCM Capacity Planning

ITM Director

agent

SCM (dashboards & analytics)

Director

TPC

… Servers, storage, network

IBM (hybrid and non-hybrid) and non-IBM Platforms (distributed and z)

TADDM

SCM Dashboards

IT System Admin

Capacity PlannerOperations

Displays TADDM

views in context

(config changes)

Platform mgmt tools (Virtual Center, IBM Systems Director, DFM, etc)

User Interface (DoJo, etc)

TCR / TIP

(WAS)

DashboardsCognos

TDW

TPC agent

ITM/ITMfVE

(monitoring server)

NMA agentNetAppagent

Hyper-V

agent

Reports

Capacity PlanningData provider

Configuration, metrics

Data Mart

Events

SCM Reports

SCM Client ( Browser)

ITM OS

agents

TEP

Citrix

Agents

KVM

Agent

VMware

agent

ITM/ITMfVE Agents

SCM

LiC (drill-down)

LiC(drill-down)

sensors

ITPA (predictive)

Forward

dynamic

changes

Import DLAs

Cisco UCS Agent

OMNIbusForward

Events (EIF)

Federated

SCM Capacity Planning

ITM Director

agent

SCM (dashboards & analytics)

Figure 1: Current SCM Architecture

Here is a list of key capabilities available in the latest SCM release for monitoring the health of virtual environments and cloud infrastructures.

� Visualization: o Web 2.0 health dashboards (TIP based) for VMware o TEP Workspaces available for VMware, IBM Power, KVM, Citrix XenServer, Citrix XenApp,

Citrix XenDesktop, Cisco UCS, OS (Windows, Linux, AIX, Solaris, HP-UX) and others (TPC-storage, NMA-network, NetApp-storage)

� Analytics (including Reports): o Capacity Planning for VMware



o Out of Box Reports for performance and capacity of VMware, IBM Power, and OS (e.g. Windows, Linux, AIX)

o Predictive Analytics for VMware and IBM Power with predictive alerts and reports

� Virtual Environment Monitored: o Vmware and IBM Power, with out-of-the-box best practices situations/alerts, expert advice and

workflows for detecting performance problems. DLAs for basic discovery of Vmware and IBM Power.

o New hypervisors and platforms added in 2011: Citrix XenServer, KVM, Citrix XenApp, Citrix XenDesktop, Cisco UCS

o NetApp (storage) physical volumes associated with the virtual storages (VMware only) o Monitoring status of physical switches and ports associated with hypervisors (VMware only) o Windows, Linux, AIX, Solaris, HP-UX with out-of-the-box best practices situations/alerts and

expert advice. DLAs for basic discovery of Windows, Linux, AIX, Solaris, HP-UX

2.0 Test Scenarios

• The Agents, Dashboard, Reports and Capacity Planner components will undergo through Component Verification Test (CVT), System Verification Test (SVT), SVT Regression Test, and ISO Regression Test. All these cycles will be carried out to cover the User Stories defined in RTC-1, RTC-2.

• To ensure the functionality and stability of the product, testing will be carried out on all the supported platforms. A separate section below describes the planned test environment.

2.1 System Verification Test (SVT)

This section describes the set of planned SVT scenarios for the following components of Smart Cloud Monitoring: - Vmware Agent - Agent Reports - Vmware Dashboard - Vmware Capacity Planner - PowerVM Capacity Planner SVT will be using TAR and ISO images alternately during every other sprint.

2.1.1 SVT Scenario 1 – VMware Agent Verify Self Describing Agent support

Objective: Verify VMware Agent SDA support is pushed to the ITM Infrastructure. Conditions Verified:

• Verify scenario by using fresh install (silent and GUI), local upgrade, and remote upgrade of the agent as root or non-root user.

• Verify that the agent support files are automatically pushed to the ITM 6.3 Infrastructure.

2.1.2 SVT Scenario 2 – Verify VMware agent new attribute group Cluster_DRS_Faults

Objective: Verify DRS faults generated during DRS Operation



Conditions Verified:


• Verify that historical data is collected after enabling Historical Data collection for the new attribute group

• Verify new queries and workspaces.

• If ESX server/VM is disconnected/inactive, performance data should show unavailable/0/-1 and it should show static data

2.1.3 SVT Scenario 3 – HBA Performance from 2 new Storage metrics.

Objective: As a VMware Administrator I want to show the Storage Adapter and Storage path metrics in order to know the HBA performance Conditions Verified:


• Verify that historical data is collected after enabling Historical Data collection for the modified attribute groups “Server” and “Server HBA”.

• Verify modified queries

• Verify modified workspaces.

2.1.4 SVT Scenario 4 – Verify HBA Capacity for Host Fiber Channel and Host Internet SCSI.

Objective: As a VMware Administrator I want to verify server HBA capacity for Host Fiber Channel and Host Internet. Conditions Verified:


• Verify that historical data is collected after enabling Historical Data collection for the modified attribute group “Server HBA”.

• Verify modified queries

• Verify modified workspaces.

2.1.5 SVT Scenario 5 – Verify new attribute group “VM_Disk_Performance”.

Objective: As a VMware Administrator I want to know how much read and write operations are performed on disk in the given performance interval Conditions Verified:


• Verify that historical data is collected after enabling Historical Data collection for the new attribute group “VM_Disk_Performance”



• Verify new queries

• Verify new workspaces.

2.1.6 SVT Scenario 6 – Verify VMware agent new attribute group “Virtual_Machine” & “Datastore Cluster”

Objective: Verify VMware Agent Storage POD Conditions Verified:




• If ESX server/VM is disconnected/inactive, performance data should show unavailable/0/-1 and it should show static data

2.1.7 SVT Scenario 7 – Verify VMware agent new attribute group “percent free space”

Objective: Verify that as a VMware administrator I can monitor Datastore free space % at datastore cluster level Conditions Verified:




2.1.8 SVT Scenario 8 – Verify VMware agent modifications to attribute group “Datastore”

Objective: Verify that as a VMware administrator, I can monitor percent of Datastore storage consumed by snapshot Conditions Verified:




2.1.9 SVT Scenario 9 – Capture virtual center tasks that have modified a managed entity

Objective: Verify that as a VMware administrator, I can capture virtual center tasks that have modified a managed entity







2.1.10 SVT Scenario 10 – VMware Agent Affinity Test

Objective: Verify Agent affinity when v7.1 and v7.2 of VMware agent connect to the same ITM TEMS with 7.2 support files Conditions Verified:


• Verify that the older agents do not see new attributes and workspaces added for v7.2.

2.1.11 SVT Scenario 11 – VMware Agent Coexistence with other agents after upgrade

Objective: Verify that the VMware agent v7.2 can coexist with other agents (older VMware agent, other v7.2 VE agents) after the upgrade. Conditions Verified:


• Verify Co-existence with other agents. o Coexistence with DB2 agent version 6.2 o Coexistence with MSApp .net agent o Coexistence with other virtualization agents version 7.2 (interim builds) o Coexistence with another agent when both agent and agent support gets installed on same server.

2.1.12 SVT Scenario 12 – Verify VMware Agent Prereq checker results

Objective: Verify the prerequisite checker for the VMware agent 7.2 before the upgrade or fresh installation. Conditions Verified:

• Verify scenario by running the prereq checker as root and non-root user.

• Validate the results of the prereq checker by running agent installation after prereq checker has reported ‘Fail’.

• Validate the results of the prereq checker by running agent installation after prereq checker has reported ‘Pass’.

• Verify scenario by running the prereq checker on all agent supported platforms.

2.1.13 SVT Scenario 13 – VMware Agent function after ITM infrastructure upgrade



Objective: Verify that the VMware agent v7.2 continues to after the ITM infrastructure is upgraded. Conditions Verified:

• Verify scenario on all supported platforms

• Verify scenario when the Agent and ITM infrastructure are on the same machine

• Verify scenario when the Agent and ITM infrastructure are on different machines

• Verify Historical Data Collection, situations, custom workspaces, and custom queries

• Verify upgrade of ITM as root or non-root user.

2.1.14 SVT Scenario 14 – VMware Agent verification with agents and Infrastructure on one server

Objective: Verify VMware Agent testing on a test bed where TEMS, TEPS, WPA, Performance Analyzer, and S&P are all installed on one server. Conditions Verified:

• Verify Agent testing on a test bed where TEMS, TEPS, WPA, Performance Analyzer, and S&P are all installed on one server for supported platforms.

• Verify Agent testing on this test bed as root or non-root user.

2.1.15 SVT Scenario 15 – VMware Dashboard and Capacity Planner virtualization data verification

Objective: Verify that the VMware Dashboard and Capacity Planner display correct data in TIP from the virtual environment agent v7.2 (VMware) can coexist with other agents (older VMware agent, other v7.2 VE agents). Conditions Verified:


• Verify scenario by fresh install (silent and GUI) of the VMware dashboard and Capacity Planner on supported platforms.

• Verify scenario by installation of all components (VMware dashboard, VMware Capacity planner, PowerVM Capacity Planner, Reports) and prerequisites (TIP, TCR, Jazz for SM) on the same server

• Verify scenario by installation of all components (VMware dashboard, Capacity planner, Reports) and prerequisites (TIP, TCR) distributed across servers

• Verify scenario by installation of VMware and Capacity Planner in a shared TIP environment. Verify by changing the order of installation (VMware first followed by PowerVM capacity planner and then PowerVM capacity planner followed by VMware Capacity Planner )

• Verify scenario by including variations like Non-Default parameters for Capacity Planner, Remote Database for Capacity Planner etc.

• Validation of Federation Configuration involving both Homogenous (TDW in DB2) and Heterogeneous (TDW in Oracle or MS SQL). Capacity Planner database is on IFS 10.1 or 9.7.

2.1.16 SVT Scenario 16 – Verify VMware Agent Error Recovery

Objective: Verify that the VMware agent v7.2 can recover from a set of error conditions.




• Error conditions can be simulated by o Restarting vCenter o Stopping Data provider service. o Killing Agent service o Restart TDW database

•

2.1.17 SVT Scenario 17 – VMware Dashboard Error Recovery Testing

Objective: Verify that the VMware Dashboard can continue to show data if one or more of the underlying components becomes unavailable Conditions Verified:

• Error/Recovery Testing for Dashboard o Restart TEPS with TIP still running o Restart TIP o Change Password for various accounts being used o Root or non-root user



2.1.18 SVT Scenario 18 – Agent Reports Verification

Objective: Verify that the VMware, KVM, and system P agent reports. Conditions Verified:

• Verify scenario by using fresh install (silent and GUI) of the 7.2 reports on TCR 3.1

• Install as root or non-root user.

• Verify scenario by importing 7.1 reports on TCR 3.1

• Verify scenario by running TCR 2.1.1 and TCR 3.1 on the same server (using different ports)

• Validation of all reports against different TDW database.



3.0 Platform Coverage Minimum ITM version supported is ITM 6.2.2 FP4 for SCM 7.2 release. ITM 6.3 is the base framework used for all the new features that are going to be implemented in this release. Those are:

1. Self Describing Agents (SDA) 2. Install Prerequisite Checker

3.1 ITM coverage

Following ITM setups are planned to be taken for testing SCM 7.2

Sl.

No

ITM

Version TEMS TEPS TEPS DB WP Agent SY Agent

TDW

Database

1 ITM 63 Windows 2008

R2 EE 64-bit

Windows 2008

R2 EE 64-bit DB2 10.1

Windows

2008 R2 EE

64-bit

Windows

2008 R2 EE

64-bit

DB2 10.1

2 ITM 623

FP1

Windows 2008

SP2 EE 64-bit

Windows 2008

SP2 EE 64-bit DB2 9.5

Windows

2008 SP2

EE 64-bit

Windows

2008 SP2 EE

64-bit

DB2 9.5

3 ITM 623 RHEL 6 64-bit Windows 2003

SP2 32 bit SQL 2005

Windows

2003 SP2

32 bit

Windows

2003 SP2 32

bit

SQL 2005

4 ITM 622

FP8 RHEL 5 32-bit RHEL 5 32-bit DB2 9.5

RHEL 5 32-

bit

RHEL 5 32-

bit DB2 9.5

5 ITM 622

FP4 RHEL 5 64-bit AIX 6.1 64-bit Derby

RHEL 5 64-

bit

RHEL 5 64-

bit DB 9.7

6 ITM 622

FP5

Solaris 10

(SPARC) 64-bit RHEL 6 64-bit DB2 9.5

Solaris 10

(SPARC)

64-bit

Solaris 10

(SPARC) 64-

bit

Oracle 11g

7 ITM 623

FP1 AIX 6 64-bit

SUSE Linux 11

64-bit Derby

SUSE Linux

11 64-bit

SUSE Linux

11 64-bit Oracle 11g

8 ITM 63 SUSE Linux 10

32-bit

Windows 2003

EE SP2 64-bit

SQL Server

2008

Windows

2003 EE

SP2 64-bit

Windows

2003 EE SP2

64-bit

SQL Server

2008

9

ITM 622

FP5 (Z-

Linux)

Z-linux Z-linux DB2 9.7 Z-linux Z-linux DB2 9.7

10 ITM 623

(Z/OS) Z/OS

Windows 2003

SE SP2 32-bit

SQL Server

2005 SP2

Windows

2003 SE

SP2 32-bit

Windows

2003 SE SP2

32-bit

SQL Server

2005 SP2



3.2 TEMA+RTEMS Coverage

Following Agent platforms are planned to be covered during the testing. During SVT the RTEMS will also be installed on Agent machine and Agents will be connected to RTEMS.

SL No. OS Version Bitness of Machine

1

Windows

Server 2008 R2 EE 64 Bit

2

Windows

Server 2003 SP2 EE 32 Bit

3

Windows

Server 2003 R2 EE 64 Bit

4

Windows

Server 2008 SE 32 Bit

5 RHEL 6 64 Bit

6 RHEL 5 32 Bit

7 SUSE 11 64 Bit

8 SUSE 10 32 Bit

3.3 TIP/TCR Coverage

Sl. No TIP/TCR Version

Operating Systems TADFDC DB

1

TIP 3.1

Windows 64 bit

Linux 64 bit

zLinux 64 bit

2

TCR 3.1

Windows -64 bit

Linux -64 bit

AIX 64 bit

zLinux 64 bit

DB2 9.5

DB2 9.7

DB2 10.1

3.4 Capacity Planner Databases Coverage

Sl. No Datamart DB

Tivoli Warehouse DB



1

DB2 9.7 DB2 9.7

2

DB2 10.1

DB2 10.1

Oracle 11g

MS SQL 2008 R2

3.5 Web Browser Coverage

Following browsers will be covered for TIP/TCR based applications

Sl. No Web Browser & Version

TIP/TCR Application

1

Internet Explorer 8

Capacity Planner

VMWare Capacity Reports

VMWare Dashboard

2

Internet Explorer 9

Capacity Planner


VMWare Dashboard

3

Firefox 10 FSR

Capacity Planner


VMWare Dashboard

Date post:	25-Mar-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Smart Cloud Monitoring 7.2 Test Coverage Document · Capacity Planner – IBM Required Summary of...

Documents