ClearQuest XML Server with ClearCase Integration Northwest Rational User’s Group February 22, 2007...

ClearQuest XML Serverwith ClearCase Integration

ClearQuest XML Serverwith ClearCase IntegrationNorthwest Rational User’s GroupFebruary 22, 2007Northwest Rational User’s GroupFebruary 22, 2007

Frank Scholz

[email protected]

Casey Stewart

[email protected]

Frank Scholz

[email protected]

Casey Stewart

[email protected]

© 2007 Mentor Graphics Corporation

All Rights Reserved

© 2007 Mentor Graphics Corporation

All Rights Reserved

ClearQuest XML Server with ClearCase IntegrationNW Rational User’s Group – February 22, 2007© 2007 Mentor Graphics Corporation, All Rights Reserved


22

Agenda

Mentor Graphics Environment Requirements High-Level Overview Samples Architecture Details Findings ClearCase Client Demonstration



33

Mentor Graphics Environment

Product development effort is widely distributed.— 3000+ developers— 30+ development sites worldwide

ClearQuest deployment is 100% web-based. ClearQuest sits behind the corporate firewall. No ClearQuest groups. Implementation by the numbers:

120,000+350,000+1,350,000

+3,500,000

+1

stateless recordsdefects recordsnotes recordshistory recordsdatabase



44

Requirements

No client installs. No remote servers. Perform well over the WAN. Scalable. Utilize a generic interface. Enforce permission restrictions. Password encryption. Notification on errors.



55

High-Level Overview

CQ Request Manager

Database

CQ Native Client

Browser Client

ToolsWeb Servers

X

User Written Client

XML Server



66

High-Level Overview

The ClearQuest XML Interface (CQ/XML) uses industry standard communication methods and formats to communicate to the ClearQuest Request Manager.— Immediate updates to ClearQuest data.— ClearQuest business rules are enforced.— Development activity can have direct

relationships to defect data. “Industry standard communication…”

— method: platform-independent TCP sockets— format: XML



77

High-Level Overview: Features

All CQ/XML communication is done via XML.— input, output, errors, logs and configuration— W3C standard

Read, write and create ClearQuest defect records.

Performs well over the WAN. Supports RC4 password encryption. Permissions can be limited to a single user,

multiple systems or anywhere in between. Allows interactive or batch response.



88

High-Level Overview: Features

Users can run existing ClearQuest queries, including prompted queries.

Special functionality has been added to support user-specific interactivity.

ClearQuest defect histories record XML updates.

Logs are reviewable in all of the current browsers.

CQ/XML server can email users ClearQuest errors.



99

Sample: view a defect’s headline

 <ClearQuest login="cq_ user" password="* * * * * " repo="cqrepos" db="dev28"> <defect id="dev2800000048" action="view" wait="yes"> <headline/> </defect> </ClearQuest>  <ClearQuest db="dev28" login="cq_ user" cqtan="06318n000001"

client="w35d125" ip="147.34.35.125"> <defect id="dev2800000048" action="view" status="ok"> <headline>Breakout length changed when joining designs.</headline> </defect> </ClearQuest>



1010

Sample: submit a defect

<ClearQuest login="cq_ user" password="* * * * * " repo="cqrepos" db="dev28"> <defect id="0" action="submit" wait="yes"> <component>Component A1</component> <description>sample submit description</description> <headline>sample submit</headline> <product>Product A</product> <version_reported>v1</version_reported> <severity>3-Average</severity> </defect> </ClearQuest>  <ClearQuest db="dev28" login="cq_ user" cqtan="06318n000002"

client="w35d125" ip="147.34.35.125"> <defect id="0" action="submit" status="ok"> <id>dev2800007374</id> </defect> </ClearQuest>



1111

Sample: “adding” to defect fields

<ClearQuest login="cq_ user" password="* * * * * " repo="cqrepos" db="dev28"> <defect id="dev2800000048" action="modify" wait="yes"> <platform behavior="add">Linux Red Hat Ent 3 x86</platform> <description behavior="add">added 2 days to est. time</description> <time_estimate behavior="add">2</time_estimate> </defect> </ClearQuest>  <ClearQuest db="dev28" login="cq_ user" cqtan="06318n000003"

client="w35d125" ip="147.34.35.125"> <defect id="dev2800000048" action="modify" status="ok" /> </ClearQuest>



1212

Sample: prompted query

<ClearQuest login="cq_ user" password="* * * * * " repo="cqrepos" db="dev28"> <query name="Personal Queries/ DEIwPrompts" wait="yes"> <prompt1>cq_ user</prompt1> <prompt2 operator="Does Not Equal">Closed</prompt2> </query> </ClearQuest>  <ClearQuest db="dev28" login="cq_ user" cqtan="06318n000004"

client="w35d125" ip="147.34.35.125"> <query name="Personal Queries/ DEIwPrompts" status="ok"> <defect row="0"> <headline>TC01 Headline String</headline> <id>dev2800007311</id> <State>Submitted</State> </defect> <defect row="1">...</defect> <defect row="2">...</defect> </query> </ClearQuest>



1313

Architecture

ClientLive

ServerQueue

Manager

ClearQuest Request Manager

Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible



1414

Architecture

ClientLive

ServerQueue

Manager


Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible

1. accept socket

2. assign conn. id

3. fork

4. read socket



1515

Architecture

ClientLive

ServerQueue

Manager


Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible

5. pre-parse XML

6. write queue file



1616

Architecture

ClientLive

ServerQueue

Manager


Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible

7. parse XML

8. restructure XML obj



1717

Architecture

ClientLive

ServerQueue

Manager


Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible

9. login to CQ

10. run CQ commands

11. logout of CQ



1818

Architecture

ClientLive

ServerQueue

Manager


Logs

Queuesocket

XML

XML

XML

XM

L

CQ

AP

I

CQ

AP

I

Logs

XM

L

ClearQuest Web

user site

Wilsonville

system visible

user visible

12. send status

13. close socket



1919

Details: Live vs. Queued

Allows faster response for jobs that don’t require a return message.

Can run on a second machine to lighten the load on the live server.

Every successfully parsed job is written to disk.

Queue manager is a scheduled process that runs every 5 minutes to process queued jobs and clean the disk.



2020

Details: XML Parser

Since you can’t compile modules for cqperl/ratlperl, the only stand-alone parser available is XML::Mini.

— XML::Mini can get in some recursive loops so we pre-parse and reformat the XML before calling the parser.

— XML::Mini doesn’t differentiate between attributes and elements so we made all attribute names unique from element names.

— Resulting data structure is complex and differs based on the input.

— Results may not be in the same order as the input. Our users aren’t well-versed on XML so we ignored

DTDs, external entities, parameter entities, XML Schema, etc.



2121

Details: <info…> Elements “Special functionality has been added to support

user-specific interactivity.” Can return a list of personal, public or all queries. Can return login or database information.

<ClearQuest login="cq_ user" password="* * * * * " repo="cqrepos" db="dev28"> <info type="Personal Queries" wait="yes"/> </ClearQuest>  <ClearQuest db="dev28" login="cq_ user" cqtan="06318n000005"

client="w35d125" ip="147.34.35.125"> <info type="Personal Queries" status="ok"> <queryname row="0">Personal Queries/ State & Release</queryname> ... <queryname row="36">Personal Queries/ WebQueries/ Closed</queryname> </ info> </ClearQuest>



2222

Details: Miscellaneous

Encryption implemented via Crypt::RC4.— Stand-alone Perl package.— Very simple and fast.

Appending to XML logs requires overwriting the root element end.

Prompting dates was error prone and dropped from the implementation.

Wrap every ClearQuest call with an eval{} so a failure in ClearQuest doesn’t kill the entire server.



2323

Findings

Stable— No application-level failures encountered

through seven months of testing and three months of production deployment.

Scales extremely well— Under Grid testing, the CQ/XML server was

still handling requests with over 350 near-simultaneous connections.

Response time meets user requirements— defect update jobs < 3 seconds— defect view jobs < 1 second— query jobs < 1 second



2424

ClearCase Client

Why not use IBM ClearCase/ClearQuest integration?— Requires Native Clients

CCRC currently only supports CQ UCM integration— Does not work over WAN and MultiSite does

not work for us— ClearCase and ClearQuest user must have

same version of client installed— Not configurable

Most of the CC information captured in CQ is unusable

Need to Configure differently on a per VOB basis (different processes and requirements per group)



2525

ClearCase Client: Implementation Details

Uses CC Triggers to send data to CQ via XML— Triggers use ratlperl/ccperl

Allows users to Associate Defect(s) to file during checkout/checkin— Uses either Command Line prompts or GUI — CC Data: File Path, Version, User, Date,

Checkin Comment Removes Defect Associations during

uncheckout CC Uses CrmRequest hyperlink to link to

Defects



2626

DemonstrationClearQuest / ClearCase Integration via

CQ/XML



2727

Date post:	19-Jan-2016
Category:	Documents
Upload:	curtis-gilmore
View:	226 times
Download:	7 times

ClearQuest XML Server with ClearCase Integration Northwest Rational User’s Group February 22, 2007...

Documents